2024-09-17T11:24:40.080 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T11:24:40.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T11:24:40.176 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768 branch: main description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7907768' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - CEPHADM_STRAY_DAEMON log-only-match: - CEPHADM_ sha1: 7498493279982dfe87f00616198a5967475f6169 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 7498493279982dfe87f00616198a5967475f6169 extra_packages: - cephadm selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 2b698643d05f73e79c2d2888228947a84a4ac0bd owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 3649 sha1: 7498493279982dfe87f00616198a5967475f6169 sleep_before_teardown: 0 subset: 42/64 suite: orch suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_2b698643d05f73e79c2d2888228947a84a4ac0bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 2b698643d05f73e79c2d2888228947a84a4ac0bd targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOm3GCtI4V36xX+jQUw/2+VU5I9x3UWARfq90HIL5FFbRTrhops+1kmohpE/HcEQOqPe3SflC0YuPG8PpqlTsIA= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpNcYpqD3ergigEj3VipTKHhbKN/UKgMj+BND4I/CazNGmnXjs/GuEAfOQfw6pYHEOkL/WlSI9nZmyj5+I3CxQ= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_20:08:15 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T11:24:40.176 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_2b698643d05f73e79c2d2888228947a84a4ac0bd/qa; will attempt to use it 2024-09-17T11:24:40.177 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_2b698643d05f73e79c2d2888228947a84a4ac0bd/qa/tasks 2024-09-17T11:24:40.177 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T11:24:40.178 INFO:teuthology.task.internal:Checking packages... 2024-09-17T11:24:40.201 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7498493279982dfe87f00616198a5967475f6169' 2024-09-17T11:24:40.201 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T11:24:40.202 INFO:teuthology.packaging:ref: None 2024-09-17T11:24:40.202 INFO:teuthology.packaging:tag: None 2024-09-17T11:24:40.202 INFO:teuthology.packaging:branch: main 2024-09-17T11:24:40.202 INFO:teuthology.packaging:sha1: 7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:24:40.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-17T11:24:40.455 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4959.g2b698643 2024-09-17T11:24:40.456 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T11:24:40.462 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T11:24:40.462 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T11:24:40.478 INFO:teuthology.task.internal:Saving configuration 2024-09-17T11:24:40.490 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T11:24:40.496 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T11:24:40.522 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 11:18:14.763077', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOm3GCtI4V36xX+jQUw/2+VU5I9x3UWARfq90HIL5FFbRTrhops+1kmohpE/HcEQOqPe3SflC0YuPG8PpqlTsIA='} 2024-09-17T11:24:40.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 11:18:14.764309', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpNcYpqD3ergigEj3VipTKHhbKN/UKgMj+BND4I/CazNGmnXjs/GuEAfOQfw6pYHEOkL/WlSI9nZmyj5+I3CxQ='} 2024-09-17T11:24:40.545 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T11:24:40.551 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T11:24:40.551 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-17T11:24:40.552 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T11:24:40.608 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fda6cf67ac0>, signals=[15]) 2024-09-17T11:24:40.608 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T11:24:40.614 INFO:teuthology.task.internal:Opening connections... 2024-09-17T11:24:40.614 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-09-17T11:24:40.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:24:40.692 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T11:24:40.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:24:40.768 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T11:24:40.774 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-09-17T11:24:40.793 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-09-17T11:24:40.793 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-09-17T11:24:40.851 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T11:24:40.852 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-09-17T11:24:40.853 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T11:24:40.853 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T11:24:40.853 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T11:24:40.853 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-09-17T11:24:40.878 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-17T11:24:40.897 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-17T11:24:40.898 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T11:24:40.956 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-17T11:24:40.957 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T11:24:40.957 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T11:24:40.957 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T11:24:40.957 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-17T11:24:40.978 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T11:24:40.988 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T11:24:40.992 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T11:24:40.992 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-09-17T11:24:40.995 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-17T11:24:41.012 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T11:24:41.018 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T11:24:41.018 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-09-17T11:24:41.053 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-17T11:24:41.253 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T11:24:41.308 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T11:24:41.321 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T11:24:41.322 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T11:24:41.322 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T11:24:41.322 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T11:24:41.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T11:24:41.340 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T11:24:41.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T11:24:41.341 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-09-17T11:24:41.398 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-17T11:24:41.398 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-17T11:24:41.399 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-17T11:24:41.400 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-503.el9.x86_64 2024-09-17T11:24:41.401 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-503.el9.x86_64 2024-09-17T11:24:41.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-09-17T11:24:44.205 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.3 MB 00:01 2024-09-17T11:24:44.839 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-17T11:24:52.872 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-17T11:24:53.705 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-17T11:24:58.122 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T11:24:59.831 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-17T11:24:59.839 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.5 MB 00:03 2024-09-17T11:25:00.643 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-17T11:25:01.439 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 19 kB 00:00 2024-09-17T11:25:02.243 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-17T11:25:06.275 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T11:25:07.674 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T11:25:08.256 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T11:25:08.256 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T11:25:08.353 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T11:25:08.362 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-09-17T11:25:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:25:08.364 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 79 M 2024-09-17T11:25:08.364 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 126 M 2024-09-17T11:25:08.364 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:25:09.226 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 62 kB/s | 36 kB 00:00 2024-09-17T11:25:09.589 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T11:25:09.589 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T11:25:09.688 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:25:09.697 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:25:09.698 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 79 M 2024-09-17T11:25:09.699 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 126 M 2024-09-17T11:25:09.699 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:25:10.026 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T11:25:10.117 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 293 kB/s | 36 kB 00:00 2024-09-17T11:25:10.693 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-17T11:25:11.485 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-17T11:25:11.485 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:25:11.485 INFO:teuthology.orchestra.run.smithi049.stdout:Total 25 MB/s | 79 MB 00:03 2024-09-17T11:25:11.967 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:01 2024-09-17T11:25:12.199 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:25:12.627 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:25:12.627 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:25:13.647 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:25:13.647 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:25:14.969 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.9 MB/s | 29 MB 00:04 2024-09-17T11:25:15.643 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 6.1 MB/s | 35 MB 00:05 2024-09-17T11:25:15.643 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:25:15.643 INFO:teuthology.orchestra.run.smithi129.stdout:Total 13 MB/s | 79 MB 00:05 2024-09-17T11:25:16.222 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:25:16.329 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:25:16.677 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:25:16.677 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:25:16.905 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T11:25:16.993 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:25:17.624 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:25:17.624 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:25:19.872 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:25:20.030 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:25:20.047 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:25:20.677 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T11:25:20.759 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:25:23.521 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:25:23.652 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:25:25.260 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:25:25.306 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:25:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:25:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:25:30.550 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:25:33.426 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:10.452 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:10.494 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T11:26:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:26:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T11:26:11.636 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T11:26:11.637 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T11:26:11.637 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:26:11.637 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:26:11.903 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:11.977 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T11:26:12.104 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:12.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T11:26:12.104 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T11:26:12.105 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T11:26:12.105 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T11:26:12.105 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T11:26:12.105 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T11:26:12.105 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T11:26:12.166 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:12.167 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T11:26:12.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-09-17T11:26:12.200 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-09-17T11:26:12.200 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-09-17T11:26:12.200 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-09-17T11:26:12.200 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 82.el9 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8274670 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-09-17T11:26:12.201 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:26:12.202 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T11:26:12.203 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T11:26:12.203 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-508.el9.x86_64 2024-09-17T11:26:12.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T11:26:12.330 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:12.331 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T11:26:12.331 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T11:26:12.331 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T11:26:12.874 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:26:13.249 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:26:13.580 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T11:26:13.666 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.666 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T11:26:13.666 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T11:26:13.667 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T11:26:13.667 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T11:26:13.667 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T11:26:13.667 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T11:26:13.667 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T11:26:13.726 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T11:26:13.726 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T11:26:13.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-09-17T11:26:13.758 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-09-17T11:26:13.758 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-09-17T11:26:13.758 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-09-17T11:26:13.758 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 82.el9 2024-09-17T11:26:13.758 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8274670 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T11:26:13.759 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-09-17T11:26:13.760 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-09-17T11:26:13.760 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T11:26:13.760 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-09-17T11:26:13.760 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-09-17T11:26:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:26:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T11:26:13.763 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T11:26:13.763 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 5.14.0-508.el9.x86_64 2024-09-17T11:26:13.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T11:26:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-17T11:26:14.379 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T11:26:14.423 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-09-17T11:26:14.426 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-09-17T11:26:14.442 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.BwTzXDoVIw 2024-09-17T11:26:14.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BwTzXDoVIw 2024-09-17T11:26:14.510 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.BwTzXDoVIw 2024-09-17T11:26:14.656 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.BwTzXDoVIw is 6KB 2024-09-17T11:26:14.678 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.BwTzXDoVIw 2024-09-17T11:26:14.694 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T11:26:14.756 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T11:26:14.757 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T11:26:14.757 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T11:26:14.757 INFO:teuthology.orchestra.run.smithi049.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T11:26:14.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T11:26:14.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-09-17T11:26:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T11:26:15.718 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-17T11:26:15.730 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T11:26:15.746 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.xjmYmKweH8 2024-09-17T11:26:15.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xjmYmKweH8 2024-09-17T11:26:15.812 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.xjmYmKweH8 2024-09-17T11:26:15.948 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.xjmYmKweH8 is 6KB 2024-09-17T11:26:15.996 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.xjmYmKweH8 2024-09-17T11:26:16.011 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T11:26:16.073 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T11:26:16.073 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T11:26:16.073 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T11:26:16.073 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T11:26:16.074 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T11:26:16.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-09-17T11:26:44.945 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T11:26:44.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:26:44.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:26:46.258 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T11:26:46.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:26:46.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:03.489 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-17T11:27:04.769 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T11:27:12.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:27:12.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:13.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:27:13.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:15.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-17T11:27:16.829 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T11:27:27.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:27:27.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:28.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:27:28.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:30.621 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-17T11:27:45.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:27:45.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:27:47.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T11:27:48.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-09-17T11:28:02.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:28:02.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:28:02.659 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T11:28:03.003 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:28:03.003 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T11:28:03.003 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T11:28:03.018 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-508.el9.x86_64 2024-09-17T11:28:03.018 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T11:28:03.018 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T11:28:03.018 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T11:28:04.019 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T11:28:04.019 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T11:28:04.100 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-17T11:28:04.130 DEBUG:teuthology.parallel:result is None 2024-09-17T11:28:06.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:28:06.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:29:06.706 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T11:29:27.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:29:27.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:29:28.049 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-09-17T11:29:28.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:29:28.396 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T11:29:28.397 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-09-17T11:29:28.413 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-508.el9.x86_64 2024-09-17T11:29:28.414 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T11:29:28.414 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T11:29:28.414 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T11:29:29.414 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T11:29:29.415 DEBUG:teuthology.orchestra.run.smithi049:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T11:29:29.507 INFO:teuthology.orchestra.run.smithi049.stdout:ttyS1 2024-09-17T11:29:29.527 DEBUG:teuthology.parallel:result is None 2024-09-17T11:29:29.528 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T11:29:29.538 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T11:29:29.538 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T11:29:29.540 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T11:29:29.561 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T11:29:29.567 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T11:29:29.575 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T11:29:29.575 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T11:29:29.600 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T11:29:29.631 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T11:29:29.638 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T11:29:29.638 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T11:29:29.669 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T11:29:29.701 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T11:29:29.707 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T11:29:29.712 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T11:29:29.718 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T11:29:29.720 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T11:29:29.726 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T11:29:29.726 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T11:29:29.756 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T11:29:29.788 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T11:29:29.797 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T11:29:29.797 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T11:29:29.824 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T11:29:29.845 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T11:29:29.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T11:29:29.975 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T11:29:30.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T11:29:30.100 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:29:30.101 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T11:29:30.168 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T11:29:30.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T11:29:30.271 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T11:29:30.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T11:29:30.402 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:29:30.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T11:29:30.472 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-09-17T11:29:30.475 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-17T11:29:30.521 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T11:29:30.563 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T11:29:30.965 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T11:29:30.973 INFO:teuthology.task.internal:Starting timer... 2024-09-17T11:29:30.973 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T11:29:30.983 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T11:29:30.994 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T11:29:30.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-09-17T11:29:31.041 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-09-17T11:29:31.043 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-17T11:29:31.129 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-17T11:29:31.132 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T11:29:31.132 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-09-17T11:29:31.158 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-09-17T11:29:31.159 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-17T11:29:31.206 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-17T11:29:31.206 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive'} 2024-09-17T11:29:31.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-17T11:29:31.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T11:29:31.235 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-17T11:29:31.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T11:29:31.264 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T11:29:31.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-09-17T11:29:31.306 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-17T11:29:31.343 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T11:29:31.352 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T11:29:31.364 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T11:29:31.365 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi129.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T11:33:17.294 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com')] 2024-09-17T11:33:17.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:33:17.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:33:17.377 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-09-17T11:33:17.456 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-09-17T11:33:17.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:33:17.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T11:33:17.535 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T11:33:17.612 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T11:33:17.612 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T11:33:17.625 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T11:33:17.625 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T11:33:17.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T11:33:17.628 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T11:33:17.628 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T11:33:17.655 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T11:33:17.670 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T11:33:17.682 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T11:33:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-09-17T11:33:17.697 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T11:33:17.707 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-09-17T11:33:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-17T11:33:17.722 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T11:33:17.733 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-17T11:33:17.736 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T11:33:17.748 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T11:33:17.762 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T11:33:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.797 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.820 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T11:33:17.822 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T11:33:17.822 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T11:33:17.823 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.823 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.823 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.823 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T11:33:17.824 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T11:33:17.834 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T11:33:17.834 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi049.front.sepia.ceph.com 2024-09-17T11:33:17.834 DEBUG:teuthology.orchestra.run.smithi049:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T11:33:17.835 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T11:33:17.835 DEBUG:teuthology.orchestra.run.smithi129:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T11:33:18.206 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:33:18.216 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout:Removing: 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout:Removing unused dependencies: 2024-09-17T11:33:18.222 INFO:teuthology.orchestra.run.smithi049.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout:Remove 2 Packages 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout:Freed space: 5.5 M 2024-09-17T11:33:18.223 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:33:18.228 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:33:18.228 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:Removing: 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:Removing unused dependencies: 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:Remove 2 Packages 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:18.231 INFO:teuthology.orchestra.run.smithi129.stdout:Freed space: 5.5 M 2024-09-17T11:33:18.232 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:33:18.235 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:33:18.235 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:33:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:33:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:33:18.289 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:33:18.289 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:33:18.428 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:33:18.432 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:33:18.496 INFO:teuthology.orchestra.run.smithi129.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T11:33:18.509 INFO:teuthology.orchestra.run.smithi049.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T11:33:18.551 INFO:teuthology.orchestra.run.smithi129.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T11:33:18.575 INFO:teuthology.orchestra.run.smithi049.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T11:33:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T11:33:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T11:33:19.040 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T11:33:19.041 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout:Removed: 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:19.223 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout:Removed: 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:19.302 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:33:20.059 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:45 ago on Tue 17 Sep 2024 11:32:34 AM UTC. 2024-09-17T11:33:20.161 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:50 ago on Tue 17 Sep 2024 11:32:29 AM UTC. 2024-09-17T11:33:20.198 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:33:20.199 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:20.199 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:33:20.199 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:20.199 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout:Install 7 Packages 2024-09-17T11:33:20.200 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:20.217 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 5.2 M 2024-09-17T11:33:20.217 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 19 M 2024-09-17T11:33:20.217 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:33:20.304 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:33:20.305 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:20.305 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:33:20.305 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:20.305 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout:Install 7 Packages 2024-09-17T11:33:20.306 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:20.319 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 5.2 M 2024-09-17T11:33:20.320 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 19 M 2024-09-17T11:33:20.320 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:33:20.553 INFO:teuthology.orchestra.run.smithi049.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 462 kB/s | 44 kB 00:00 2024-09-17T11:33:20.595 INFO:teuthology.orchestra.run.smithi049.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 768 kB/s | 106 kB 00:00 2024-09-17T11:33:20.620 INFO:teuthology.orchestra.run.smithi049.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-17T11:33:20.653 INFO:teuthology.orchestra.run.smithi049.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-17T11:33:20.679 INFO:teuthology.orchestra.run.smithi049.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.0 MB/s | 906 kB 00:00 2024-09-17T11:33:20.729 INFO:teuthology.orchestra.run.smithi049.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.5 MB/s | 837 kB 00:00 2024-09-17T11:33:21.058 INFO:teuthology.orchestra.run.smithi129.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 74 kB/s | 44 kB 00:00 2024-09-17T11:33:21.117 INFO:teuthology.orchestra.run.smithi129.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 161 kB/s | 106 kB 00:00 2024-09-17T11:33:21.120 INFO:teuthology.orchestra.run.smithi049.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 6.8 MB/s | 3.2 MB 00:00 2024-09-17T11:33:21.121 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:33:21.121 INFO:teuthology.orchestra.run.smithi049.stdout:Total 5.7 MB/s | 5.2 MB 00:00 2024-09-17T11:33:21.233 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:33:21.249 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:33:21.250 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:33:21.351 INFO:teuthology.orchestra.run.smithi129.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.0 MB/s | 906 kB 00:00 2024-09-17T11:33:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:33:21.561 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:33:21.734 INFO:teuthology.orchestra.run.smithi129.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 106 kB/s | 72 kB 00:00 2024-09-17T11:33:21.826 INFO:teuthology.orchestra.run.smithi129.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 119 kB/s | 84 kB 00:00 2024-09-17T11:33:22.043 INFO:teuthology.orchestra.run.smithi129.stdout:(6/7): runc-1.1.13-4.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-17T11:33:22.058 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:33:22.149 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T11:33:22.184 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T11:33:22.211 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T11:33:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 912 kB/s | 837 kB 00:00 2024-09-17T11:33:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:33:22.278 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.6 MB/s | 5.2 MB 00:01 2024-09-17T11:33:22.392 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:33:22.408 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:33:22.408 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:33:22.434 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T11:33:22.469 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T11:33:22.728 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:33:22.728 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:33:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T11:33:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T11:33:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T11:33:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:23.201 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:33:23.295 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-17T11:33:23.339 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-17T11:33:23.367 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-17T11:33:23.573 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-17T11:33:23.607 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T11:33:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-17T11:33:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T11:33:24.126 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:24.126 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T11:33:24.127 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T11:33:24.127 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T11:33:24.127 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T11:33:24.127 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T11:33:24.127 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T11:33:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-17T11:33:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:24.538 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:24.539 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:33:24.928 DEBUG:teuthology.parallel:result is None 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-17T11:33:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-17T11:33:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-17T11:33:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:25.743 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:33:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T11:33:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T11:33:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T11:33:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-17T11:33:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:25.744 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:33:26.124 DEBUG:teuthology.parallel:result is None 2024-09-17T11:33:26.124 INFO:teuthology.run_tasks:Running task install... 2024-09-17T11:33:26.132 DEBUG:teuthology.task.install:project ceph 2024-09-17T11:33:26.132 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}, 'extra_packages': ['cephadm']} 2024-09-17T11:33:26.132 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7498493279982dfe87f00616198a5967475f6169'} 2024-09-17T11:33:26.132 INFO:teuthology.task.install:Using flavor: default 2024-09-17T11:33:26.138 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T11:33:26.138 INFO:teuthology.task.install:extra packages: [] 2024-09-17T11:33:26.138 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7498493279982dfe87f00616198a5967475f6169', 'tag': None, 'wait_for_package': False} 2024-09-17T11:33:26.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:33:26.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7498493279982dfe87f00616198a5967475f6169', 'tag': None, 'wait_for_package': False} 2024-09-17T11:33:26.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:33:26.281 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/ 2024-09-17T11:33:26.281 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4937.g74984932 2024-09-17T11:33:26.291 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/ 2024-09-17T11:33:26.292 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4937.g74984932 2024-09-17T11:33:26.391 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T11:33:26.391 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:33:26.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T11:33:26.402 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/7498493279982dfe87f00616198a5967475f6169/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T11:33:26.402 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:33:26.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T11:33:26.426 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T11:33:26.426 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7498493279982dfe87f00616198a5967475f6169/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T11:33:26.429 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T11:33:26.429 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7498493279982dfe87f00616198a5967475f6169/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T11:33:26.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T11:33:26.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T11:33:26.570 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T11:33:26.577 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T11:33:26.636 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-17T11:33:26.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-17T11:33:26.643 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-09-17T11:33:26.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-09-17T11:33:26.862 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-17T11:33:26.877 INFO:teuthology.orchestra.run.smithi049.stdout:45 files removed 2024-09-17T11:33:26.889 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2024-09-17T11:33:26.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw 2024-09-17T11:33:27.603 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 195 kB/s | 84 kB 00:00 2024-09-17T11:33:27.629 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 198 kB/s | 84 kB 00:00 2024-09-17T11:33:27.895 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-17T11:33:27.929 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-17T11:33:28.171 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T11:33:28.222 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-17T11:33:28.629 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-17T11:33:30.955 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.3 MB 00:02 2024-09-17T11:33:33.206 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-09-17T11:33:36.414 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-17T11:33:38.690 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-17T11:33:40.382 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-17T11:33:40.698 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-17T11:33:41.495 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-17T11:33:42.324 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-17T11:33:43.453 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-17T11:33:46.785 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T11:33:49.015 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:33:49.017 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:33:49.017 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:49.017 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:33:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.3.0-4937.g74984932.el9 ceph 11 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.5 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.2 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.3.0-4937.g74984932.el9 ceph 5.5 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.3.0-4937.g74984932.el9 ceph 22 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.3.0-4937.g74984932.el9 ceph 25 k 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T11:33:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 755 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.3.0-4937.g74984932.el9 ceph 508 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T11:33:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 5.5 M 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.3.0-4937.g74984932.el9 ceph 45 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.3.0-4937.g74984932.el9 ceph 143 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.3.0-4937.g74984932.el9 ceph 162 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.3.0-4937.g74984932.el9 ceph 321 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.3.0-4937.g74984932.el9 ceph 305 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.3.0-4937.g74984932.el9 ceph 100 k 2024-09-17T11:33:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:33:49.021 INFO:teuthology.orchestra.run.smithi129.stdout:Install 31 Packages 2024-09-17T11:33:49.022 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 2 Packages 2024-09-17T11:33:49.022 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:49.022 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 62 M 2024-09-17T11:33:49.022 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:33:49.050 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T11:33:49.736 INFO:teuthology.orchestra.run.smithi129.stdout:(1/33): ceph-base-19.3.0-4937.g74984932.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-17T11:33:49.778 INFO:teuthology.orchestra.run.smithi129.stdout:(2/33): ceph-selinux-19.3.0-4937.g74984932.el9. 607 kB/s | 25 kB 00:00 2024-09-17T11:33:49.878 INFO:teuthology.orchestra.run.smithi129.stdout:(3/33): libcephfs2-19.3.0-4937.g74984932.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-17T11:33:49.954 INFO:teuthology.orchestra.run.smithi129.stdout:(4/33): libradosstriper1-19.3.0-4937.g74984932. 6.6 MB/s | 508 kB 00:00 2024-09-17T11:33:50.362 INFO:teuthology.orchestra.run.smithi129.stdout:(5/33): ceph-common-19.3.0-4937.g74984932.el9.x 20 MB/s | 22 MB 00:01 2024-09-17T11:33:50.446 INFO:teuthology.orchestra.run.smithi129.stdout:(6/33): librgw2-19.3.0-4937.g74984932.el9.x86_6 11 MB/s | 5.5 MB 00:00 2024-09-17T11:33:50.471 INFO:teuthology.orchestra.run.smithi129.stdout:(7/33): python3-ceph-argparse-19.3.0-4937.g7498 414 kB/s | 45 kB 00:00 2024-09-17T11:33:50.522 INFO:teuthology.orchestra.run.smithi129.stdout:(8/33): python3-ceph-common-19.3.0-4937.g749849 1.9 MB/s | 143 kB 00:00 2024-09-17T11:33:50.547 INFO:teuthology.orchestra.run.smithi129.stdout:(9/33): python3-cephfs-19.3.0-4937.g74984932.el 2.1 MB/s | 162 kB 00:00 2024-09-17T11:33:50.572 INFO:teuthology.orchestra.run.smithi129.stdout:(10/33): python3-rados-19.3.0-4937.g74984932.el 6.2 MB/s | 321 kB 00:00 2024-09-17T11:33:50.598 INFO:teuthology.orchestra.run.smithi129.stdout:(11/33): python3-rbd-19.3.0-4937.g74984932.el9. 5.9 MB/s | 305 kB 00:00 2024-09-17T11:33:50.623 INFO:teuthology.orchestra.run.smithi129.stdout:(12/33): python3-rgw-19.3.0-4937.g74984932.el9. 1.9 MB/s | 100 kB 00:00 2024-09-17T11:33:50.757 INFO:teuthology.orchestra.run.smithi129.stdout:(13/33): ceph-radosgw-19.3.0-4937.g74984932.el9 7.3 MB/s | 11 MB 00:01 2024-09-17T11:33:50.799 INFO:teuthology.orchestra.run.smithi129.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 533 kB/s | 107 kB 00:00 2024-09-17T11:33:50.832 INFO:teuthology.orchestra.run.smithi129.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 817 kB/s | 171 kB 00:00 2024-09-17T11:33:50.857 INFO:teuthology.orchestra.run.smithi129.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-17T11:33:50.899 INFO:teuthology.orchestra.run.smithi129.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-17T11:33:50.925 INFO:teuthology.orchestra.run.smithi129.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-17T11:33:50.950 INFO:teuthology.orchestra.run.smithi129.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-09-17T11:33:50.984 INFO:teuthology.orchestra.run.smithi129.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-17T11:33:51.009 INFO:teuthology.orchestra.run.smithi129.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 613 kB/s | 36 kB 00:00 2024-09-17T11:33:51.034 INFO:teuthology.orchestra.run.smithi129.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-17T11:33:51.060 INFO:teuthology.orchestra.run.smithi129.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-17T11:33:51.118 INFO:teuthology.orchestra.run.smithi129.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-17T11:33:51.143 INFO:teuthology.orchestra.run.smithi129.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-17T11:33:51.169 INFO:teuthology.orchestra.run.smithi129.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 275 kB/s | 26 kB 00:00 2024-09-17T11:33:51.191 INFO:teuthology.orchestra.run.smithi129.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-17T11:33:51.233 INFO:teuthology.orchestra.run.smithi129.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T11:33:51.283 INFO:teuthology.orchestra.run.smithi129.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 839 kB 00:00 2024-09-17T11:33:51.342 INFO:teuthology.orchestra.run.smithi129.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-17T11:33:51.361 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:33:51.363 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:51.363 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:33:51.363 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.3.0-4937.g74984932.el9 ceph 11 M 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.5 M 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.2 M 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.3.0-4937.g74984932.el9 ceph 5.5 M 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.3.0-4937.g74984932.el9 ceph 22 M 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.3.0-4937.g74984932.el9 ceph 25 k 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T11:33:51.364 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 755 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.3.0-4937.g74984932.el9 ceph 508 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.3.0-4937.g74984932.el9 ceph 5.5 M 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T11:33:51.365 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.3.0-4937.g74984932.el9 ceph 45 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.3.0-4937.g74984932.el9 ceph 143 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.3.0-4937.g74984932.el9 ceph 162 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.3.0-4937.g74984932.el9 ceph 321 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.3.0-4937.g74984932.el9 ceph 305 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.3.0-4937.g74984932.el9 ceph 100 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T11:33:51.366 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Install 31 Packages 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 2 Packages 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 62 M 2024-09-17T11:33:51.367 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:33:51.417 INFO:teuthology.orchestra.run.smithi129.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-17T11:33:51.567 INFO:teuthology.orchestra.run.smithi129.stdout:(32/33): librbd1-19.3.0-4937.g74984932.el9.x86_ 14 MB/s | 3.2 MB 00:00 2024-09-17T11:33:51.626 INFO:teuthology.orchestra.run.smithi129.stdout:(33/33): librados2-19.3.0-4937.g74984932.el9.x8 10 MB/s | 3.5 MB 00:00 2024-09-17T11:33:51.631 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:33:51.631 INFO:teuthology.orchestra.run.smithi129.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-17T11:33:51.742 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:33:51.778 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:33:51.778 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:33:52.251 INFO:teuthology.orchestra.run.smithi049.stdout:(1/33): ceph-base-19.3.0-4937.g74984932.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-17T11:33:52.295 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:33:52.295 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:33:52.335 INFO:teuthology.orchestra.run.smithi049.stdout:(2/33): ceph-selinux-19.3.0-4937.g74984932.el9. 303 kB/s | 25 kB 00:00 2024-09-17T11:33:52.427 INFO:teuthology.orchestra.run.smithi049.stdout:(3/33): libcephfs2-19.3.0-4937.g74984932.el9.x8 8.1 MB/s | 755 kB 00:00 2024-09-17T11:33:52.594 INFO:teuthology.orchestra.run.smithi049.stdout:(4/33): ceph-radosgw-19.3.0-4937.g74984932.el9. 14 MB/s | 11 MB 00:00 2024-09-17T11:33:52.627 INFO:teuthology.orchestra.run.smithi049.stdout:(5/33): libradosstriper1-19.3.0-4937.g74984932. 2.5 MB/s | 508 kB 00:00 2024-09-17T11:33:52.686 INFO:teuthology.orchestra.run.smithi049.stdout:(6/33): python3-ceph-argparse-19.3.0-4937.g7498 774 kB/s | 45 kB 00:00 2024-09-17T11:33:52.769 INFO:teuthology.orchestra.run.smithi049.stdout:(7/33): python3-ceph-common-19.3.0-4937.g749849 1.7 MB/s | 143 kB 00:00 2024-09-17T11:33:52.853 INFO:teuthology.orchestra.run.smithi049.stdout:(8/33): librgw2-19.3.0-4937.g74984932.el9.x86_6 21 MB/s | 5.5 MB 00:00 2024-09-17T11:33:52.878 INFO:teuthology.orchestra.run.smithi049.stdout:(9/33): python3-cephfs-19.3.0-4937.g74984932.el 1.5 MB/s | 162 kB 00:00 2024-09-17T11:33:52.903 INFO:teuthology.orchestra.run.smithi049.stdout:(10/33): python3-rados-19.3.0-4937.g74984932.el 6.3 MB/s | 321 kB 00:00 2024-09-17T11:33:52.928 INFO:teuthology.orchestra.run.smithi049.stdout:(11/33): python3-rbd-19.3.0-4937.g74984932.el9. 6.0 MB/s | 305 kB 00:00 2024-09-17T11:33:52.971 INFO:teuthology.orchestra.run.smithi049.stdout:(12/33): python3-rgw-19.3.0-4937.g74984932.el9. 1.5 MB/s | 100 kB 00:00 2024-09-17T11:33:53.188 INFO:teuthology.orchestra.run.smithi049.stdout:(13/33): ceph-common-19.3.0-4937.g74984932.el9. 16 MB/s | 22 MB 00:01 2024-09-17T11:33:53.271 INFO:teuthology.orchestra.run.smithi049.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-09-17T11:33:53.322 INFO:teuthology.orchestra.run.smithi049.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 486 kB/s | 171 kB 00:00 2024-09-17T11:33:53.347 INFO:teuthology.orchestra.run.smithi049.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-17T11:33:53.372 INFO:teuthology.orchestra.run.smithi049.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-09-17T11:33:53.423 INFO:teuthology.orchestra.run.smithi049.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-17T11:33:53.447 INFO:teuthology.orchestra.run.smithi049.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-17T11:33:53.473 INFO:teuthology.orchestra.run.smithi049.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-17T11:33:53.498 INFO:teuthology.orchestra.run.smithi049.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 712 kB/s | 36 kB 00:00 2024-09-17T11:33:53.524 INFO:teuthology.orchestra.run.smithi049.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-17T11:33:53.549 INFO:teuthology.orchestra.run.smithi049.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-17T11:33:53.607 INFO:teuthology.orchestra.run.smithi049.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-17T11:33:53.633 INFO:teuthology.orchestra.run.smithi049.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-17T11:33:53.658 INFO:teuthology.orchestra.run.smithi049.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-09-17T11:33:53.683 INFO:teuthology.orchestra.run.smithi049.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T11:33:53.717 INFO:teuthology.orchestra.run.smithi049.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-17T11:33:53.764 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:33:53.777 INFO:teuthology.orchestra.run.smithi049.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-17T11:33:53.826 INFO:teuthology.orchestra.run.smithi049.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-17T11:33:53.851 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T11:33:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T11:33:53.918 INFO:teuthology.orchestra.run.smithi049.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T11:33:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T11:33:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T11:33:53.971 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T11:33:54.010 INFO:teuthology.orchestra.run.smithi049.stdout:(32/33): librados2-19.3.0-4937.g74984932.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-17T11:33:54.068 INFO:teuthology.orchestra.run.smithi049.stdout:(33/33): librbd1-19.3.0-4937.g74984932.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-17T11:33:54.073 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:33:54.074 INFO:teuthology.orchestra.run.smithi049.stdout:Total 23 MB/s | 62 MB 00:02 2024-09-17T11:33:54.166 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T11:33:54.193 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:33:54.227 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T11:33:54.234 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:33:54.234 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:33:54.587 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T11:33:54.700 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:33:54.782 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:33:54.782 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:33:54.793 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:33:54.883 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 8/35 2024-09-17T11:33:54.919 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 9/35 2024-09-17T11:33:54.965 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 9/35 2024-09-17T11:33:55.002 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T11:33:55.054 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.3.0-4937.g74984932.el9. 11/35 2024-09-17T11:33:55.133 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 12/35 2024-09-17T11:33:55.154 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 13/35 2024-09-17T11:33:55.218 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 13/35 2024-09-17T11:33:55.253 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T11:33:55.365 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T11:33:55.404 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T11:33:55.484 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T11:33:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T11:33:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T11:33:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T11:33:55.636 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T11:33:55.958 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T11:33:56.004 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.3.0-4937.g74984932.el9.x86_64 23/35 2024-09-17T11:33:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.3.0-4937.g74984932.el9.x86_64 23/35 2024-09-17T11:33:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 24/35 2024-09-17T11:33:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T11:33:56.266 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:33:56.369 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T11:33:56.408 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T11:33:56.449 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T11:33:56.471 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T11:33:56.496 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T11:33:56.674 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T11:33:56.691 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T11:33:56.706 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.3.0-4937.g74984932.el9.x86_64 27/35 2024-09-17T11:33:56.753 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T11:33:56.778 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.3.0-4937.g74984932.el9.x86_64 27/35 2024-09-17T11:33:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 28/35 2024-09-17T11:33:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.3.0-4937.g74984932.el9.x8 29/35 2024-09-17T11:33:57.116 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T11:33:57.234 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:33:57.338 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:33:57.428 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 8/35 2024-09-17T11:33:57.461 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 9/35 2024-09-17T11:33:57.507 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 9/35 2024-09-17T11:33:57.536 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T11:33:57.602 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.3.0-4937.g74984932.el9. 11/35 2024-09-17T11:33:57.675 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 12/35 2024-09-17T11:33:57.704 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 13/35 2024-09-17T11:33:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 13/35 2024-09-17T11:33:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T11:33:57.897 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T11:33:57.929 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T11:33:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T11:33:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T11:33:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T11:33:58.073 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T11:33:58.111 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T11:33:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T11:33:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.3.0-4937.g74984932.el9.x86_64 23/35 2024-09-17T11:33:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.3.0-4937.g74984932.el9.x86_64 23/35 2024-09-17T11:33:58.616 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 24/35 2024-09-17T11:33:58.661 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T11:33:59.178 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T11:33:59.206 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.3.0-4937.g74984932.el9.x86_64 27/35 2024-09-17T11:33:59.208 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:33:59.248 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:33:59.277 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.3.0-4937.g74984932.el9.x86_64 27/35 2024-09-17T11:33:59.358 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 28/35 2024-09-17T11:33:59.383 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.3.0-4937.g74984932.el9.x8 29/35 2024-09-17T11:33:59.808 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:33:59.843 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 31/35 2024-09-17T11:33:59.920 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 31/35 2024-09-17T11:33:59.920 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T11:33:59.920 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T11:33:59.920 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:33:59.978 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:01.859 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:34:01.896 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:34:02.492 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 30/35 2024-09-17T11:34:02.524 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 31/35 2024-09-17T11:34:02.601 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 31/35 2024-09-17T11:34:02.601 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T11:34:02.602 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T11:34:02.602 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:02.634 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:09.797 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-17T11:34:09.798 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:10.827 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 33/35 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 33/35 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T11:34:10.852 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:10.854 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T11:34:10.929 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T11:34:10.929 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 1/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 3/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 4/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 5/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 6/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:34:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4937.g74984932.el9. 8/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.3.0-4937.g74984932.el9.x8 9/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 10/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 11/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 12/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 13/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T11:34:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T11:34:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T11:34:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.3.0-4937.g74984932.el9.x86_64 34/35 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.952 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T11:34:12.953 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T11:34:12.954 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T11:34:12.955 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T11:34:12.955 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T11:34:12.955 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:12.955 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:13.132 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:13.132 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-09-17T11:34:13.132 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-09-17T11:34:13.132 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-09-17T11:34:13.132 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-09-17T11:34:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-09-17T11:34:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-09-17T11:34:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-09-17T11:34:13.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:13.351 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2024-09-17T11:34:13.932 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:14.062 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.3.0-4937.g74984932.el9 ceph 45 M 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.3.0-4937.g74984932.el9 ceph 161 k 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T11:34:14.063 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T11:34:14.064 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 46 M 2024-09-17T11:34:14.065 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 201 M 2024-09-17T11:34:14.065 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:14.183 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 33/35 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 33/35 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T11:34:14.210 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:14.212 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T11:34:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T11:34:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:14.401 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): libcephsqlite-19.3.0-4937.g74984932.el9. 686 kB/s | 161 kB 00:00 2024-09-17T11:34:14.426 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-17T11:34:14.559 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2024-09-17T11:34:15.470 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 1/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 3/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 4/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 5/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_6 6/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.3.0-4937.g74984932.el9.x86_64 7/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4937.g74984932.el9. 8/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.3.0-4937.g74984932.el9.x8 9/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 10/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 11/35 2024-09-17T11:34:15.471 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 12/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 13/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T11:34:15.472 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.3.0-4937.g74984932.el9.x86_64 32/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T11:34:15.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.3.0-4937.g74984932.el9.x86_64 34/35 2024-09-17T11:34:15.760 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): ceph-test-19.3.0-4937.g74984932.el9.x86_ 29 MB/s | 45 MB 00:01 2024-09-17T11:34:15.762 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:15.762 INFO:teuthology.orchestra.run.smithi129.stdout:Total 27 MB/s | 46 MB 00:01 2024-09-17T11:34:15.789 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:15.810 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:15.810 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:16.164 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:16.165 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:16.402 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T11:34:16.403 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T11:34:16.404 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T11:34:16.405 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:16.406 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:16.808 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T11:34:16.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-test 2024-09-17T11:34:16.858 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T11:34:16.887 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 3/4 2024-09-17T11:34:17.410 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:17.546 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:17.546 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.3.0-4937.g74984932.el9 ceph 45 M 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.3.0-4937.g74984932.el9 ceph 161 k 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-09-17T11:34:17.547 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:17.548 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 46 M 2024-09-17T11:34:17.548 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 201 M 2024-09-17T11:34:17.548 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:17.909 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): libcephsqlite-19.3.0-4937.g74984932.el9. 731 kB/s | 161 kB 00:00 2024-09-17T11:34:17.934 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-17T11:34:17.951 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T11:34:19.160 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): ceph-test-19.3.0-4937.g74984932.el9.x86_ 31 MB/s | 45 MB 00:01 2024-09-17T11:34:19.161 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:19.162 INFO:teuthology.orchestra.run.smithi049.stdout:Total 29 MB/s | 46 MB 00:01 2024-09-17T11:34:19.190 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:34:19.212 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:34:19.213 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:34:19.584 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:34:19.584 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:34:20.183 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:34:20.274 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T11:34:20.325 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T11:34:20.353 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 3/4 2024-09-17T11:34:21.255 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 3/4 2024-09-17T11:34:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 4/4 2024-09-17T11:34:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 4/4 2024-09-17T11:34:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 1/4 2024-09-17T11:34:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 2/4 2024-09-17T11:34:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T11:34:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T11:34:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:24.070 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:24.349 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2024-09-17T11:34:24.947 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:24.961 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 3/4 2024-09-17T11:34:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 4/4 2024-09-17T11:34:25.092 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:25.094 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.3.0-4937.g74984932.el9 ceph 6.6 k 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.3.0-4937.g74984932.el9 ceph 2.4 M 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.3.0-4937.g74984932.el9 ceph 1.5 M 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 291 k 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.3.0-4937.g74984932.el9 ceph 4.8 M 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.3.0-4937.g74984932.el9 ceph 17 M 2024-09-17T11:34:25.095 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T11:34:25.096 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T11:34:25.097 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:25.098 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout:Install 37 Packages 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 31 M 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 109 M 2024-09-17T11:34:25.099 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:25.468 INFO:teuthology.orchestra.run.smithi129.stdout:(1/37): ceph-19.3.0-4937.g74984932.el9.x86_64.r 44 kB/s | 6.6 kB 00:00 2024-09-17T11:34:25.626 INFO:teuthology.orchestra.run.smithi129.stdout:(2/37): ceph-mgr-19.3.0-4937.g74984932.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-17T11:34:25.668 INFO:teuthology.orchestra.run.smithi129.stdout:(3/37): ceph-mds-19.3.0-4937.g74984932.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-17T11:34:25.760 INFO:teuthology.orchestra.run.smithi129.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4937.g7498 3.1 MB/s | 291 kB 00:00 2024-09-17T11:34:25.894 INFO:teuthology.orchestra.run.smithi129.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-17T11:34:25.977 INFO:teuthology.orchestra.run.smithi129.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-09-17T11:34:26.019 INFO:teuthology.orchestra.run.smithi129.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-17T11:34:26.061 INFO:teuthology.orchestra.run.smithi129.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-17T11:34:26.103 INFO:teuthology.orchestra.run.smithi129.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-09-17T11:34:26.145 INFO:teuthology.orchestra.run.smithi129.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 5.0 MB/s | 215 kB 00:00 2024-09-17T11:34:26.339 INFO:teuthology.orchestra.run.smithi129.stdout:(11/37): ceph-osd-19.3.0-4937.g74984932.el9.x86 24 MB/s | 17 MB 00:00 2024-09-17T11:34:26.413 INFO:teuthology.orchestra.run.smithi129.stdout:(12/37): ceph-mon-19.3.0-4937.g74984932.el9.x86 5.1 MB/s | 4.8 MB 00:00 2024-09-17T11:34:26.471 INFO:teuthology.orchestra.run.smithi129.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 579 kB/s | 188 kB 00:00 2024-09-17T11:34:26.497 INFO:teuthology.orchestra.run.smithi129.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2024-09-17T11:34:26.522 INFO:teuthology.orchestra.run.smithi129.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 936 kB/s | 172 kB 00:00 2024-09-17T11:34:26.572 INFO:teuthology.orchestra.run.smithi129.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 766 kB/s | 77 kB 00:00 2024-09-17T11:34:26.598 INFO:teuthology.orchestra.run.smithi129.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-09-17T11:34:26.623 INFO:teuthology.orchestra.run.smithi129.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 412 kB/s | 42 kB 00:00 2024-09-17T11:34:26.674 INFO:teuthology.orchestra.run.smithi129.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T11:34:26.699 INFO:teuthology.orchestra.run.smithi129.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 579 kB/s | 43 kB 00:00 2024-09-17T11:34:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-17T11:34:26.749 INFO:teuthology.orchestra.run.smithi129.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-17T11:34:26.774 INFO:teuthology.orchestra.run.smithi129.stdout:(23/37): lua-devel-5.4.4-4.el9.x86_64.rpm 110 kB/s | 22 kB 00:00 2024-09-17T11:34:26.800 INFO:teuthology.orchestra.run.smithi129.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-17T11:34:26.834 INFO:teuthology.orchestra.run.smithi129.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 742 kB/s | 19 kB 00:00 2024-09-17T11:34:26.859 INFO:teuthology.orchestra.run.smithi129.stdout:(26/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-17T11:34:26.884 INFO:teuthology.orchestra.run.smithi129.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 406 kB/s | 20 kB 00:00 2024-09-17T11:34:26.910 INFO:teuthology.orchestra.run.smithi129.stdout:(28/37): python3-jaraco-collections-3.0.0-8.el9 172 kB/s | 23 kB 00:00 2024-09-17T11:34:26.935 INFO:teuthology.orchestra.run.smithi129.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-17T11:34:26.960 INFO:teuthology.orchestra.run.smithi129.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T11:34:26.985 INFO:teuthology.orchestra.run.smithi129.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-17T11:34:27.010 INFO:teuthology.orchestra.run.smithi129.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T11:34:27.036 INFO:teuthology.orchestra.run.smithi129.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-09-17T11:34:27.069 INFO:teuthology.orchestra.run.smithi129.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-17T11:34:27.094 INFO:teuthology.orchestra.run.smithi129.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-17T11:34:27.120 INFO:teuthology.orchestra.run.smithi129.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-17T11:34:27.145 INFO:teuthology.orchestra.run.smithi129.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-17T11:34:27.151 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:27.151 INFO:teuthology.orchestra.run.smithi129.stdout:Total 15 MB/s | 31 MB 00:02 2024-09-17T11:34:27.304 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:27.345 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:27.345 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:27.472 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 4/4 2024-09-17T11:34:27.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 1/4 2024-09-17T11:34:27.473 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 2/4 2024-09-17T11:34:27.474 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:27.749 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T11:34:27.750 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T11:34:27.750 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:27.750 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:27.789 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:27.789 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:27.999 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph 2024-09-17T11:34:28.594 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:28.732 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:28.734 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:28.734 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:34:28.734 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:28.734 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.3.0-4937.g74984932.el9 ceph 6.6 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.3.0-4937.g74984932.el9 ceph 2.4 M 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.3.0-4937.g74984932.el9 ceph 1.5 M 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 291 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.3.0-4937.g74984932.el9 ceph 4.8 M 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.3.0-4937.g74984932.el9 ceph 17 M 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T11:34:28.735 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T11:34:28.736 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T11:34:28.737 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout:Install 37 Packages 2024-09-17T11:34:28.738 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:28.739 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 31 M 2024-09-17T11:34:28.739 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 109 M 2024-09-17T11:34:28.739 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:29.148 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T11:34:29.181 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T11:34:29.217 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T11:34:29.291 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T11:34:29.344 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T11:34:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T11:34:29.403 INFO:teuthology.orchestra.run.smithi049.stdout:(1/37): ceph-19.3.0-4937.g74984932.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-17T11:34:29.476 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T11:34:29.554 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T11:34:29.595 INFO:teuthology.orchestra.run.smithi049.stdout:(2/37): ceph-mgr-19.3.0-4937.g74984932.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-17T11:34:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T11:34:29.645 INFO:teuthology.orchestra.run.smithi049.stdout:(3/37): ceph-mds-19.3.0-4937.g74984932.el9.x86_ 6.5 MB/s | 2.4 MB 00:00 2024-09-17T11:34:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T11:34:29.675 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T11:34:29.710 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T11:34:29.737 INFO:teuthology.orchestra.run.smithi049.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4937.g7498 3.1 MB/s | 291 kB 00:00 2024-09-17T11:34:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T11:34:29.801 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T11:34:29.804 INFO:teuthology.orchestra.run.smithi049.stdout:(5/37): ceph-mon-19.3.0-4937.g74984932.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-17T11:34:29.845 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T11:34:29.887 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T11:34:29.944 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T11:34:29.976 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T11:34:30.067 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T11:34:30.171 INFO:teuthology.orchestra.run.smithi049.stdout:(6/37): ceph-osd-19.3.0-4937.g74984932.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-17T11:34:30.204 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T11:34:30.285 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T11:34:30.337 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T11:34:30.384 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T11:34:30.421 INFO:teuthology.orchestra.run.smithi049.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 370 kB/s | 253 kB 00:00 2024-09-17T11:34:30.444 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T11:34:30.508 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T11:34:30.538 INFO:teuthology.orchestra.run.smithi049.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T11:34:30.630 INFO:teuthology.orchestra.run.smithi049.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-17T11:34:30.655 INFO:teuthology.orchestra.run.smithi049.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.0 MB/s | 125 kB 00:00 2024-09-17T11:34:30.696 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T11:34:30.741 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T11:34:30.773 INFO:teuthology.orchestra.run.smithi049.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-17T11:34:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-17T11:34:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T11:34:30.831 INFO:teuthology.orchestra.run.smithi049.stdout:(13/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-17T11:34:30.856 INFO:teuthology.orchestra.run.smithi049.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-17T11:34:30.881 INFO:teuthology.orchestra.run.smithi049.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 461 kB/s | 35 kB 00:00 2024-09-17T11:34:30.907 INFO:teuthology.orchestra.run.smithi049.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T11:34:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T11:34:30.932 INFO:teuthology.orchestra.run.smithi049.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-17T11:34:30.957 INFO:teuthology.orchestra.run.smithi049.stdout:(18/37): python3-packaging-20.9-5.el9.noarch.rp 613 kB/s | 77 kB 00:00 2024-09-17T11:34:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T11:34:31.008 INFO:teuthology.orchestra.run.smithi049.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-17T11:34:31.033 INFO:teuthology.orchestra.run.smithi049.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-17T11:34:31.058 INFO:teuthology.orchestra.run.smithi049.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-09-17T11:34:31.092 INFO:teuthology.orchestra.run.smithi049.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-17T11:34:31.093 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T11:34:31.117 INFO:teuthology.orchestra.run.smithi049.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-17T11:34:31.142 INFO:teuthology.orchestra.run.smithi049.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 368 kB/s | 18 kB 00:00 2024-09-17T11:34:31.168 INFO:teuthology.orchestra.run.smithi049.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 933 kB/s | 23 kB 00:00 2024-09-17T11:34:31.193 INFO:teuthology.orchestra.run.smithi049.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-17T11:34:31.218 INFO:teuthology.orchestra.run.smithi049.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 797 kB/s | 20 kB 00:00 2024-09-17T11:34:31.243 INFO:teuthology.orchestra.run.smithi049.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 928 kB/s | 46 kB 00:00 2024-09-17T11:34:31.269 INFO:teuthology.orchestra.run.smithi049.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T11:34:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9. 32/37 2024-09-17T11:34:31.294 INFO:teuthology.orchestra.run.smithi049.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T11:34:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 33/37 2024-09-17T11:34:31.319 INFO:teuthology.orchestra.run.smithi049.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T11:34:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 33/37 2024-09-17T11:34:31.329 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:31.329 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T11:34:31.329 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T11:34:31.329 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T11:34:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:31.344 INFO:teuthology.orchestra.run.smithi049.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T11:34:31.370 INFO:teuthology.orchestra.run.smithi049.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T11:34:31.395 INFO:teuthology.orchestra.run.smithi049.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-17T11:34:31.420 INFO:teuthology.orchestra.run.smithi049.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-17T11:34:31.445 INFO:teuthology.orchestra.run.smithi049.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T11:34:31.471 INFO:teuthology.orchestra.run.smithi049.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-17T11:34:31.474 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:31.474 INFO:teuthology.orchestra.run.smithi049.stdout:Total 11 MB/s | 31 MB 00:02 2024-09-17T11:34:31.634 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:34:31.678 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:34:31.678 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:34:32.174 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:34:32.175 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:34:32.794 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 34/37 2024-09-17T11:34:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 34/37 2024-09-17T11:34:32.822 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:32.822 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T11:34:32.822 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T11:34:32.822 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T11:34:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 35/37 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 35/37 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T11:34:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:33.486 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:34:33.537 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 36/37 2024-09-17T11:34:33.548 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T11:34:33.561 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 36/37 2024-09-17T11:34:33.561 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:33.561 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T11:34:33.562 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T11:34:33.562 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T11:34:33.562 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:33.582 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T11:34:33.609 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T11:34:33.626 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.3.0-4937.g74984932.el9.x86_64 37/37 2024-09-17T11:34:33.675 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T11:34:33.725 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T11:34:33.758 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T11:34:33.853 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T11:34:33.930 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T11:34:33.986 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T11:34:34.021 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T11:34:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T11:34:34.085 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T11:34:34.118 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T11:34:34.159 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T11:34:34.211 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T11:34:34.245 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T11:34:34.294 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T11:34:34.334 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T11:34:34.410 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T11:34:34.577 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T11:34:34.661 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T11:34:34.712 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T11:34:34.750 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T11:34:34.808 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T11:34:34.872 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T11:34:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T11:34:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T11:34:35.193 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T11:34:35.282 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T11:34:35.331 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T11:34:35.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T11:34:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-2:19.3.0-4937.g74984932.el9.x86_64 37/37 2024-09-17T11:34:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.3.0-4937.g74984932.el9.x86_64 1/37 2024-09-17T11:34:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 2/37 2024-09-17T11:34:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 3/37 2024-09-17T11:34:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 4/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 5/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9. 6/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T11:34:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T11:34:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T11:34:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T11:34:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T11:34:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T11:34:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T11:34:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T11:34:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T11:34:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T11:34:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T11:34:35.663 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9. 32/37 2024-09-17T11:34:35.688 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 33/37 2024-09-17T11:34:35.714 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 33/37 2024-09-17T11:34:35.715 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:35.715 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T11:34:35.715 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T11:34:35.715 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T11:34:35.715 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:36.425 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T11:34:36.426 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T11:34:36.427 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:36.428 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:36.872 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-base 2024-09-17T11:34:37.272 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 34/37 2024-09-17T11:34:37.304 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 34/37 2024-09-17T11:34:37.305 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:37.305 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T11:34:37.305 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T11:34:37.305 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T11:34:37.305 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:37.456 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:37.545 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:34:37.587 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:37.588 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:34:37.588 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:37.668 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install cephadm 2024-09-17T11:34:37.773 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 35/37 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 35/37 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T11:34:37.798 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:38.040 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 36/37 2024-09-17T11:34:38.065 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 36/37 2024-09-17T11:34:38.066 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:38.066 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T11:34:38.066 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T11:34:38.066 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T11:34:38.066 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:38.123 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.3.0-4937.g74984932.el9.x86_64 37/37 2024-09-17T11:34:38.245 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:38.376 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:38.377 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:34:38.377 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:38.377 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 784 k 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 784 k 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 791 k 2024-09-17T11:34:38.378 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:38.663 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm-19.3.0-4937.g74984932.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-17T11:34:38.664 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:38.664 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-17T11:34:38.664 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:38.669 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:38.669 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:38.675 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:38.675 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:38.748 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:39.209 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:39.669 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-2:19.3.0-4937.g74984932.el9.x86_64 37/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.3.0-4937.g74984932.el9.x86_64 1/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 2/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 3/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 4/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 5/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9. 6/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T11:34:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T11:34:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T11:34:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T11:34:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T11:34:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T11:34:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T11:34:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T11:34:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T11:34:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T11:34:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:39.926 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:40.197 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T11:34:40.780 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:40.795 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T11:34:40.796 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T11:34:40.797 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T11:34:40.798 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T11:34:40.799 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:40.799 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:40.909 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4937.g74984932.el9 ceph 145 k 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:34:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:40.911 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 145 k 2024-09-17T11:34:40.911 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 432 k 2024-09-17T11:34:40.911 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:41.105 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-immutable-object-cache-19.3.0-4937.g749849 747 kB/s | 145 kB 00:00 2024-09-17T11:34:41.106 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:41.106 INFO:teuthology.orchestra.run.smithi129.stdout:Total 740 kB/s | 145 kB 00:00 2024-09-17T11:34:41.106 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:41.114 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:41.114 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:41.148 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-base 2024-09-17T11:34:41.171 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:41.172 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:41.406 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:41.431 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:41.431 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:41.431 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T11:34:41.431 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:41.766 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:41.858 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-base-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:34:41.902 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:41.903 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:34:41.904 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:41.983 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install cephadm 2024-09-17T11:34:42.423 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:42.423 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:42.423 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:42.423 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:42.424 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:42.424 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:42.581 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:42.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2024-09-17T11:34:42.720 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 784 k 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:34:42.721 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:42.722 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 784 k 2024-09-17T11:34:42.722 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 791 k 2024-09-17T11:34:42.722 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:43.007 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm-19.3.0-4937.g74984932.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-17T11:34:43.008 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:43.008 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-17T11:34:43.008 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:34:43.014 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:34:43.014 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:34:43.020 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:34:43.020 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:34:43.095 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:34:43.268 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:43.359 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:34:43.404 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:43.405 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:34:43.405 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:43.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T11:34:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:44.062 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:44.195 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:44.196 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:44.196 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:34:44.196 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 14 M 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 34 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 16 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T11:34:44.197 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:44.198 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 19 M 2024-09-17T11:34:44.199 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 159 M 2024-09-17T11:34:44.199 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:44.291 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:44.504 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T11:34:44.527 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4937.g74 245 kB/s | 34 kB 00:00 2024-09-17T11:34:44.552 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4937.g749 100 kB/s | 16 kB 00:00 2024-09-17T11:34:44.761 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-17T11:34:44.878 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-17T11:34:45.037 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): ceph-mgr-dashboard-19.3.0-4937.g7498493 22 MB/s | 14 MB 00:00 2024-09-17T11:34:45.070 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.8 MB/s | 548 kB 00:00 2024-09-17T11:34:45.095 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-17T11:34:45.113 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:45.137 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-17T11:34:45.171 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.1 MB/s | 31 kB 00:00 2024-09-17T11:34:45.196 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-17T11:34:45.238 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-17T11:34:45.249 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4937.g74984932.el9 ceph 145 k 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:45.251 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:34:45.251 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:45.251 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 145 k 2024-09-17T11:34:45.251 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 432 k 2024-09-17T11:34:45.251 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:45.454 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-immutable-object-cache-19.3.0-4937.g749849 714 kB/s | 145 kB 00:00 2024-09-17T11:34:45.455 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:45.455 INFO:teuthology.orchestra.run.smithi049.stdout:Total 709 kB/s | 145 kB 00:00 2024-09-17T11:34:45.455 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:34:45.464 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:34:45.464 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:34:45.525 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:34:45.525 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:34:45.687 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:34:45.739 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:45.765 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:45.765 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:34:45.765 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T11:34:45.765 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:45.905 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 755 kB/s | 863 kB 00:01 2024-09-17T11:34:45.909 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:45.910 INFO:teuthology.orchestra.run.smithi129.stdout:Total 11 MB/s | 19 MB 00:01 2024-09-17T11:34:45.958 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:45.977 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:45.977 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:46.252 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:46.252 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:46.705 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4937.g74984932. 1/1 2024-09-17T11:34:46.706 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:46.706 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:46.706 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:34:46.706 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:46.706 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:46.844 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:46.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr 2024-09-17T11:34:46.930 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T11:34:46.974 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T11:34:47.020 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T11:34:47.060 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T11:34:47.162 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T11:34:47.302 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T11:34:47.396 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T11:34:47.448 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T11:34:47.477 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T11:34:47.509 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9 10/12 2024-09-17T11:34:47.519 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:47.611 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-mgr-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:34:47.657 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:47.658 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:34:47.658 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:47.732 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T11:34:48.337 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:48.476 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 14 M 2024-09-17T11:34:48.477 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 34 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 16 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:48.478 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout:Install 12 Packages 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 19 M 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 159 M 2024-09-17T11:34:48.479 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:49.275 INFO:teuthology.orchestra.run.smithi049.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4937.g74 234 kB/s | 34 kB 00:00 2024-09-17T11:34:49.300 INFO:teuthology.orchestra.run.smithi049.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4937.g749 96 kB/s | 16 kB 00:00 2024-09-17T11:34:49.467 INFO:teuthology.orchestra.run.smithi049.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-09-17T11:34:49.517 INFO:teuthology.orchestra.run.smithi049.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-09-17T11:34:49.584 INFO:teuthology.orchestra.run.smithi049.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 8.1 MB/s | 548 kB 00:00 2024-09-17T11:34:49.609 INFO:teuthology.orchestra.run.smithi049.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-17T11:34:49.685 INFO:teuthology.orchestra.run.smithi049.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-17T11:34:49.709 INFO:teuthology.orchestra.run.smithi049.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-17T11:34:49.735 INFO:teuthology.orchestra.run.smithi049.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-17T11:34:49.760 INFO:teuthology.orchestra.run.smithi049.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-17T11:34:49.936 INFO:teuthology.orchestra.run.smithi049.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4937.g749849 18 MB/s | 14 MB 00:00 2024-09-17T11:34:49.969 INFO:teuthology.orchestra.run.smithi049.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.7 MB/s | 863 kB 00:00 2024-09-17T11:34:49.972 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:49.972 INFO:teuthology.orchestra.run.smithi049.stdout:Total 13 MB/s | 19 MB 00:01 2024-09-17T11:34:50.024 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:34:50.045 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:34:50.045 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:34:50.337 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:34:50.337 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:34:50.690 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el 11/12 2024-09-17T11:34:50.748 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 12/12 2024-09-17T11:34:50.945 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:34:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T11:34:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T11:34:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T11:34:51.157 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T11:34:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T11:34:51.397 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T11:34:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T11:34:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T11:34:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T11:34:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9 10/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 12/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el 1/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 2/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9 3/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T11:34:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T11:34:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T11:34:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:53.050 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:34:53.353 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T11:34:53.927 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:07 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:34:54.063 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:34:54.064 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T11:34:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:34:54.064 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 7.4 M 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T11:34:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:34:54.066 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2024-09-17T11:34:54.067 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 212 M 2024-09-17T11:34:54.067 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:34:54.297 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-17T11:34:54.372 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-09-17T11:34:54.414 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-17T11:34:54.490 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 243 kB/s | 18 kB 00:00 2024-09-17T11:34:54.582 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-17T11:34:54.607 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-17T11:34:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-17T11:34:54.782 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-09-17T11:34:54.833 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-17T11:34:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el 11/12 2024-09-17T11:34:55.118 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 12/12 2024-09-17T11:34:55.341 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-17T11:34:56.025 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.4 MB/s | 4.9 MB 00:01 2024-09-17T11:34:56.158 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.5 MB/s | 5.1 MB 00:01 2024-09-17T11:34:56.159 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:34:56.160 INFO:teuthology.orchestra.run.smithi129.stdout:Total 18 MB/s | 38 MB 00:02 2024-09-17T11:34:56.509 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:34:56.535 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:34:56.535 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 12/12 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el 1/12 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noa 2/12 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9 3/12 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T11:34:56.841 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T11:34:56.842 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T11:34:56.956 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:34:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:34:57.175 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:57.176 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:34:57.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T11:34:57.732 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:34:57.803 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T11:34:57.848 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T11:34:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T11:34:57.991 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T11:34:58.014 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:34:58.152 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 7.4 M 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T11:34:58.154 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout:Install 12 Packages 2024-09-17T11:34:58.155 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:34:58.156 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 38 M 2024-09-17T11:34:58.156 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 212 M 2024-09-17T11:34:58.156 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:34:58.176 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T11:34:58.207 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T11:34:58.283 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T11:34:58.558 INFO:teuthology.orchestra.run.smithi049.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 958 kB/s | 191 kB 00:00 2024-09-17T11:34:58.675 INFO:teuthology.orchestra.run.smithi049.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-09-17T11:34:58.709 INFO:teuthology.orchestra.run.smithi049.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-17T11:34:58.826 INFO:teuthology.orchestra.run.smithi049.stdout:(4/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-17T11:34:58.909 INFO:teuthology.orchestra.run.smithi049.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 91 kB/s | 18 kB 00:00 2024-09-17T11:34:58.943 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T11:34:58.951 INFO:teuthology.orchestra.run.smithi049.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-17T11:34:59.052 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T11:34:59.085 INFO:teuthology.orchestra.run.smithi049.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-09-17T11:34:59.135 INFO:teuthology.orchestra.run.smithi049.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-17T11:34:59.252 INFO:teuthology.orchestra.run.smithi049.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-17T11:34:59.419 INFO:teuthology.orchestra.run.smithi049.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-17T11:34:59.836 INFO:teuthology.orchestra.run.smithi049.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 5.3 MB/s | 4.9 MB 00:00 2024-09-17T11:35:00.304 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T11:35:01.003 INFO:teuthology.orchestra.run.smithi049.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.3 MB/s | 16 MB 00:01 2024-09-17T11:35:01.003 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:01.004 INFO:teuthology.orchestra.run.smithi049.stdout:Total 13 MB/s | 38 MB 00:02 2024-09-17T11:35:01.371 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:01.372 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T11:35:01.398 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:01.398 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:01.408 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 12/12 2024-09-17T11:35:01.851 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:01.852 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:02.789 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:02.865 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T11:35:02.910 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T11:35:03.016 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T11:35:03.053 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T11:35:03.236 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T11:35:03.260 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T11:35:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T11:35:04.060 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T11:35:04.079 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 12/12 2024-09-17T11:35:04.079 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 1/12 2024-09-17T11:35:04.079 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T11:35:04.079 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T11:35:04.080 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T11:35:04.186 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T11:35:04.533 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:04.534 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:04.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2024-09-17T11:35:05.502 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T11:35:05.506 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:05.640 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 49 k 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:35:05.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:05.642 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout:Install 13 Packages 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 M 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 26 M 2024-09-17T11:35:05.643 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:06.029 INFO:teuthology.orchestra.run.smithi129.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 348 kB/s | 26 kB 00:00 2024-09-17T11:35:06.054 INFO:teuthology.orchestra.run.smithi129.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 194 kB/s | 19 kB 00:00 2024-09-17T11:35:06.138 INFO:teuthology.orchestra.run.smithi129.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-17T11:35:06.163 INFO:teuthology.orchestra.run.smithi129.stdout:(4/13): ceph-mgr-rook-19.3.0-4937.g74984932.el9 237 kB/s | 49 kB 00:00 2024-09-17T11:35:06.197 INFO:teuthology.orchestra.run.smithi129.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-17T11:35:06.239 INFO:teuthology.orchestra.run.smithi129.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 776 kB/s | 32 kB 00:00 2024-09-17T11:35:06.264 INFO:teuthology.orchestra.run.smithi129.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 534 kB/s | 54 kB 00:00 2024-09-17T11:35:06.289 INFO:teuthology.orchestra.run.smithi129.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-17T11:35:06.323 INFO:teuthology.orchestra.run.smithi129.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-17T11:35:06.365 INFO:teuthology.orchestra.run.smithi129.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-17T11:35:06.390 INFO:teuthology.orchestra.run.smithi129.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 904 kB/s | 59 kB 00:00 2024-09-17T11:35:06.415 INFO:teuthology.orchestra.run.smithi129.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-17T11:35:06.449 INFO:teuthology.orchestra.run.smithi129.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-17T11:35:06.453 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:06.453 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-17T11:35:06.516 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:06.531 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:06.531 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:06.592 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T11:35:06.621 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:06.622 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:06.625 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 12/12 2024-09-17T11:35:07.126 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:07.277 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T11:35:07.359 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T11:35:07.423 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T11:35:07.469 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T11:35:07.509 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T11:35:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T11:35:07.728 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T11:35:07.795 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T11:35:08.515 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T11:35:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T11:35:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T11:35:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T11:35:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 13/13 2024-09-17T11:35:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 12/12 2024-09-17T11:35:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984 1/12 2024-09-17T11:35:09.107 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T11:35:09.108 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T11:35:09.463 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 13/13 2024-09-17T11:35:09.463 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 1/13 2024-09-17T11:35:09.463 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T11:35:09.463 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T11:35:09.464 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T11:35:09.465 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T11:35:09.465 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T11:35:09.465 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T11:35:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T11:35:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T11:35:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T11:35:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T11:35:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T11:35:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T11:35:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T11:35:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T11:35:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T11:35:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T11:35:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:09.514 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T11:35:09.805 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T11:35:09.806 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T11:35:09.807 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T11:35:09.807 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:09.807 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:09.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-rook 2024-09-17T11:35:10.076 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T11:35:10.478 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:21 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:10.620 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:35:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:10.620 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 49 k 2024-09-17T11:35:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T11:35:10.621 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T11:35:10.622 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T11:35:10.623 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:10.623 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:10.623 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:10.623 INFO:teuthology.orchestra.run.smithi049.stdout:Install 13 Packages 2024-09-17T11:35:10.623 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:10.624 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 2.2 M 2024-09-17T11:35:10.624 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 26 M 2024-09-17T11:35:10.625 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:10.658 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:24 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:10.792 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:10.793 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:10.793 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:35:10.793 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:10.793 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 154 k 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout:Install 6 Packages 2024-09-17T11:35:10.794 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:10.795 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.0 M 2024-09-17T11:35:10.795 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 33 M 2024-09-17T11:35:10.795 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:11.084 INFO:teuthology.orchestra.run.smithi049.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 206 kB/s | 26 kB 00:00 2024-09-17T11:35:11.109 INFO:teuthology.orchestra.run.smithi049.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 128 kB/s | 19 kB 00:00 2024-09-17T11:35:11.134 INFO:teuthology.orchestra.run.smithi049.stdout:(3/13): ceph-mgr-rook-19.3.0-4937.g74984932.el9 281 kB/s | 49 kB 00:00 2024-09-17T11:35:11.229 INFO:teuthology.orchestra.run.smithi129.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 2.0 MB/s | 248 kB 00:00 2024-09-17T11:35:11.251 INFO:teuthology.orchestra.run.smithi049.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-17T11:35:11.268 INFO:teuthology.orchestra.run.smithi049.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-17T11:35:11.288 INFO:teuthology.orchestra.run.smithi129.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-17T11:35:11.310 INFO:teuthology.orchestra.run.smithi049.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 913 kB/s | 54 kB 00:00 2024-09-17T11:35:11.313 INFO:teuthology.orchestra.run.smithi129.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-17T11:35:11.335 INFO:teuthology.orchestra.run.smithi049.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-17T11:35:11.338 INFO:teuthology.orchestra.run.smithi129.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4937.g74984932.e 667 kB/s | 154 kB 00:00 2024-09-17T11:35:11.364 INFO:teuthology.orchestra.run.smithi129.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-17T11:35:11.369 INFO:teuthology.orchestra.run.smithi049.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-17T11:35:11.404 INFO:teuthology.orchestra.run.smithi049.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 157 kB/s | 14 kB 00:00 2024-09-17T11:35:11.428 INFO:teuthology.orchestra.run.smithi049.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-17T11:35:11.453 INFO:teuthology.orchestra.run.smithi049.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-17T11:35:11.478 INFO:teuthology.orchestra.run.smithi049.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.6 MB/s | 220 kB 00:00 2024-09-17T11:35:11.512 INFO:teuthology.orchestra.run.smithi049.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-17T11:35:11.514 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:11.514 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.5 MB/s | 2.2 MB 00:00 2024-09-17T11:35:11.580 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:11.597 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:11.597 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:11.700 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:11.701 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T11:35:12.547 INFO:teuthology.orchestra.run.smithi129.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-17T11:35:12.549 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:12.549 INFO:teuthology.orchestra.run.smithi129.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-09-17T11:35:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T11:35:12.620 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:12.631 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:12.631 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T11:35:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T11:35:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T11:35:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T11:35:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T11:35:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T11:35:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:13.187 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T11:35:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T11:35:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T11:35:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T11:35:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T11:35:13.860 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T11:35:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T11:35:13.913 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T11:35:13.937 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T11:35:13.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 13/13 2024-09-17T11:35:13.975 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 6/6 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 13/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 1/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T11:35:14.726 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T11:35:14.727 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T11:35:14.727 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T11:35:14.727 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T11:35:14.727 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T11:35:14.727 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T11:35:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 6/6 2024-09-17T11:35:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/6 2024-09-17T11:35:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T11:35:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T11:35:14.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T11:35:14.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T11:35:15.079 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T11:35:15.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:15.081 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:15.162 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T11:35:15.162 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:15.162 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:15.163 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:15.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T11:35:15.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2024-09-17T11:35:15.952 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:16.034 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:16.093 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:16.093 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:16.093 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 154 k 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:16.094 INFO:teuthology.orchestra.run.smithi049.stdout:Install 6 Packages 2024-09-17T11:35:16.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:16.095 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 7.0 M 2024-09-17T11:35:16.095 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 33 M 2024-09-17T11:35:16.095 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:16.169 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:16.170 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.3.0-4937.g74984932.el9 ceph 900 k 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:16.171 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 980 k 2024-09-17T11:35:16.172 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 2.9 M 2024-09-17T11:35:16.172 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:16.430 INFO:teuthology.orchestra.run.smithi049.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-17T11:35:16.456 INFO:teuthology.orchestra.run.smithi049.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4937.g74984932.e 742 kB/s | 154 kB 00:00 2024-09-17T11:35:16.506 INFO:teuthology.orchestra.run.smithi049.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-17T11:35:16.531 INFO:teuthology.orchestra.run.smithi049.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 769 kB/s | 58 kB 00:00 2024-09-17T11:35:16.556 INFO:teuthology.orchestra.run.smithi049.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-17T11:35:16.618 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 475 kB/s | 80 kB 00:00 2024-09-17T11:35:16.707 INFO:teuthology.orchestra.run.smithi049.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-17T11:35:16.709 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:16.709 INFO:teuthology.orchestra.run.smithi049.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-17T11:35:16.759 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): ceph-fuse-19.3.0-4937.g74984932.el9.x86_ 2.8 MB/s | 900 kB 00:00 2024-09-17T11:35:16.760 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:16.761 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.6 MB/s | 980 kB 00:00 2024-09-17T11:35:16.773 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:16.782 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:16.794 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:16.795 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:16.795 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:16.795 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:16.877 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:16.878 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:16.902 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:16.902 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:17.068 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:17.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T11:35:17.225 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:17.292 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2/2 2024-09-17T11:35:17.406 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T11:35:17.460 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T11:35:18.017 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T11:35:18.034 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2/2 2024-09-17T11:35:18.034 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/2 2024-09-17T11:35:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T11:35:18.183 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T11:35:18.213 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 6/6 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:18.346 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:18.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-volume 2024-09-17T11:35:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 6/6 2024-09-17T11:35:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 1/6 2024-09-17T11:35:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T11:35:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T11:35:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T11:35:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T11:35:19.259 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T11:35:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:19.314 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:19.393 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:19.393 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 285 k 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 285 k 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.3 M 2024-09-17T11:35:19.394 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:19.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-fuse 2024-09-17T11:35:19.607 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-volume-19.3.0-4937.g74984932.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-17T11:35:19.608 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:19.608 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-17T11:35:19.609 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:19.617 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:19.617 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:19.646 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:19.647 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:19.864 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:19.935 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:19.950 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:19.951 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:35:19.951 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T11:35:19.951 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:20.185 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:20.324 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.3.0-4937.g74984932.el9 ceph 900 k 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:20.325 INFO:teuthology.orchestra.run.smithi049.stdout:Install 2 Packages 2024-09-17T11:35:20.326 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:20.326 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 980 k 2024-09-17T11:35:20.326 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 2.9 M 2024-09-17T11:35:20.326 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:20.492 INFO:teuthology.orchestra.run.smithi049.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 670 kB/s | 80 kB 00:00 2024-09-17T11:35:20.659 INFO:teuthology.orchestra.run.smithi049.stdout:(2/2): ceph-fuse-19.3.0-4937.g74984932.el9.x86_ 3.1 MB/s | 900 kB 00:00 2024-09-17T11:35:20.659 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:20.660 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.9 MB/s | 980 kB 00:00 2024-09-17T11:35:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:20.695 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:20.695 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:20.781 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:20.781 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:20.983 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:21.004 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:21.124 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T11:35:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2/2 2024-09-17T11:35:21.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados-devel 2024-09-17T11:35:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2/2 2024-09-17T11:35:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/2 2024-09-17T11:35:21.936 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:22.068 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:22.069 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:22.069 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T11:35:22.069 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.3.0-4937.g74984932.el9 ceph 127 k 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:22.070 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:22.071 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 127 k 2024-09-17T11:35:22.071 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 456 k 2024-09-17T11:35:22.071 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:22.125 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T11:35:22.126 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:22.126 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:22.126 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.3.0-4937.g74984932.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T11:35:22.126 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:22.126 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:22.280 INFO:teuthology.orchestra.run.smithi129.stdout:librados-devel-19.3.0-4937.g74984932.el9.x86_64 604 kB/s | 127 kB 00:00 2024-09-17T11:35:22.281 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:22.281 INFO:teuthology.orchestra.run.smithi129.stdout:Total 599 kB/s | 127 kB 00:00 2024-09-17T11:35:22.281 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:22.286 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:22.287 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:22.358 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:22.359 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:22.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-volume 2024-09-17T11:35:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:22.611 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:23.018 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:23.098 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:23.157 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:23.157 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:23.158 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-09-17T11:35:23.158 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:23.158 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:23.158 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.3.0-4937.g74984932.el9 ceph-noarch 285 k 2024-09-17T11:35:23.159 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:23.159 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:23.159 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:23.159 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:23.159 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:23.160 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 285 k 2024-09-17T11:35:23.160 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 1.3 M 2024-09-17T11:35:23.160 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:23.348 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:23.397 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-volume-19.3.0-4937.g74984932.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-17T11:35:23.398 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:23.398 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T11:35:23.398 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:23.406 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:23.407 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:23.436 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:23.436 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:23.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2024-09-17T11:35:23.671 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:23.741 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:23.757 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:23.757 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:35:23.757 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T11:35:23.757 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:24.255 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:24.345 INFO:teuthology.orchestra.run.smithi129.stdout:Package libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:24.391 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:24.392 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:24.392 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:24.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 1/1 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.3.0-4937.g74984932.el9.noarch 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:24.770 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:25.048 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:25.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados-devel 2024-09-17T11:35:25.183 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.3.0-4937.g74984932.el9 ceph 32 k 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 32 k 2024-09-17T11:35:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 142 k 2024-09-17T11:35:25.185 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:25.335 INFO:teuthology.orchestra.run.smithi129.stdout:libcephfs-devel-19.3.0-4937.g74984932.el9.x86_6 211 kB/s | 32 kB 00:00 2024-09-17T11:35:25.335 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:25.336 INFO:teuthology.orchestra.run.smithi129.stdout:Total 210 kB/s | 32 kB 00:00 2024-09-17T11:35:25.336 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:25.339 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:25.339 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:25.357 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:25.358 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:25.437 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:25.548 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:25.682 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:25.822 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.3.0-4937.g74984932.el9 ceph 127 k 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:25.823 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:25.824 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:25.824 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:25.824 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 127 k 2024-09-17T11:35:25.824 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 456 k 2024-09-17T11:35:25.824 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:25.941 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:26.026 INFO:teuthology.orchestra.run.smithi049.stdout:librados-devel-19.3.0-4937.g74984932.el9.x86_64 627 kB/s | 127 kB 00:00 2024-09-17T11:35:26.026 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:26.026 INFO:teuthology.orchestra.run.smithi049.stdout:Total 623 kB/s | 127 kB 00:00 2024-09-17T11:35:26.027 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:26.033 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:26.033 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:26.106 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:26.106 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:26.168 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:26.268 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:26.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2024-09-17T11:35:26.360 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:26.847 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:26.941 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:27.030 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:27.073 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:27.074 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:27.074 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:27.101 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:27.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2024-09-17T11:35:27.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs2 2024-09-17T11:35:27.738 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:27.827 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:27.870 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:27.871 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:27.871 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:27.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rados 2024-09-17T11:35:28.022 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:28.115 INFO:teuthology.orchestra.run.smithi049.stdout:Package libcephfs2-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:28.162 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:28.163 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:28.163 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:28.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs-devel 2024-09-17T11:35:28.524 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:28.614 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:28.658 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:28.659 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:28.659 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:28.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rgw 2024-09-17T11:35:28.845 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:28.986 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.3.0-4937.g74984932.el9 ceph 32 k 2024-09-17T11:35:28.987 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 32 k 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 142 k 2024-09-17T11:35:28.988 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:29.138 INFO:teuthology.orchestra.run.smithi049.stdout:libcephfs-devel-19.3.0-4937.g74984932.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-17T11:35:29.138 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:29.138 INFO:teuthology.orchestra.run.smithi049.stdout:Total 210 kB/s | 32 kB 00:00 2024-09-17T11:35:29.138 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:29.142 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:29.142 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:29.166 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:29.166 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:29.266 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:29.323 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:29.371 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:29.414 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:29.457 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:29.458 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:29.458 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:29.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-cephfs 2024-09-17T11:35:29.741 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:29.969 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:30.114 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:30.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados2 2024-09-17T11:35:30.204 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:30.247 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:30.248 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:30.249 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:30.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rbd 2024-09-17T11:35:30.809 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:30.902 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:30.909 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:30.948 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:30.949 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:30.949 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:31.000 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:31.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librbd1 2024-09-17T11:35:31.043 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:31.044 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T11:35:31.044 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:31.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2024-09-17T11:35:31.646 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:31.707 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:31.738 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:31.785 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:31.786 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:31.786 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:31.841 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:31.841 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:31.841 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T11:35:31.841 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.3.0-4937.g74984932.el9 ceph 85 k 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 85 k 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 231 k 2024-09-17T11:35:31.842 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:31.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rados 2024-09-17T11:35:32.063 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-fuse-19.3.0-4937.g74984932.el9.x86_64.rpm 386 kB/s | 85 kB 00:00 2024-09-17T11:35:32.064 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:32.064 INFO:teuthology.orchestra.run.smithi129.stdout:Total 383 kB/s | 85 kB 00:00 2024-09-17T11:35:32.064 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:32.071 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:32.071 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:32.136 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:32.136 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:32.328 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:32.438 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:32.470 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:32.564 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rados-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:32.610 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:32.611 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:32.612 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:32.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rgw 2024-09-17T11:35:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:33.219 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:33.295 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:33.386 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rgw-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:33.432 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:33.433 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:33.433 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:33.443 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-mirror 2024-09-17T11:35:33.513 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-cephfs 2024-09-17T11:35:34.045 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:34.126 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:34.182 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.2 M 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.2 M 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 14 M 2024-09-17T11:35:34.183 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:34.220 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-cephfs-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:34.267 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:34.268 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:34.268 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:34.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rbd 2024-09-17T11:35:34.502 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-mirror-19.3.0-4937.g74984932.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-17T11:35:34.502 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:34.502 INFO:teuthology.orchestra.run.smithi129.stdout:Total 10 MB/s | 3.2 MB 00:00 2024-09-17T11:35:34.503 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:34.512 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:34.512 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:34.594 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:34.594 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:34.964 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:35.043 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:35.057 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rbd-2:19.3.0-4937.g74984932.el9.x86_64 is already installed. 2024-09-17T11:35:35.104 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:35.105 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-09-17T11:35:35.105 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:35.119 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:35.145 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:35:35.146 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T11:35:35.146 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T11:35:35.146 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T11:35:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:35.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-fuse 2024-09-17T11:35:35.790 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:35.930 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.3.0-4937.g74984932.el9 ceph 85 k 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:35.931 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:35.932 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:35.932 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:35.932 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 85 k 2024-09-17T11:35:35.932 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 231 k 2024-09-17T11:35:35.932 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:36.124 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-fuse-19.3.0-4937.g74984932.el9.x86_64.rpm 444 kB/s | 85 kB 00:00 2024-09-17T11:35:36.125 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:36.125 INFO:teuthology.orchestra.run.smithi049.stdout:Total 441 kB/s | 85 kB 00:00 2024-09-17T11:35:36.125 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:36.132 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:36.132 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:36.200 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:36.201 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:36.219 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:36.219 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:36.219 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:36.219 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:36.220 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:36.220 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:36.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-17T11:35:36.499 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:37.101 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 11:33:46 AM UTC. 2024-09-17T11:35:37.150 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:37.150 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:37.150 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:37.150 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:37.151 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:37.151 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:37.235 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T11:35:37.235 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:37.235 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.3.0-4937.g74984932.el9 ceph 171 k 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T11:35:37.236 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:37.237 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 171 k 2024-09-17T11:35:37.237 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 490 k 2024-09-17T11:35:37.237 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T11:35:37.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-mirror 2024-09-17T11:35:37.448 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-nbd-19.3.0-4937.g74984932.el9.x86_64.rpm 810 kB/s | 171 kB 00:00 2024-09-17T11:35:37.449 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:37.449 INFO:teuthology.orchestra.run.smithi129.stdout:Total 804 kB/s | 171 kB 00:00 2024-09-17T11:35:37.449 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T11:35:37.456 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T11:35:37.456 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T11:35:37.522 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T11:35:37.523 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T11:35:37.678 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T11:35:37.787 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:38.026 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:38.165 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:38.165 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.3.0-4937.g74984932.el9 ceph 3.2 M 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:38.166 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 3.2 M 2024-09-17T11:35:38.167 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 14 M 2024-09-17T11:35:38.167 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:38.245 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:38.497 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T11:35:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:35:38.498 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T11:35:38.519 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-mirror-19.3.0-4937.g74984932.el9.x86_64.rpm 9.1 MB/s | 3.2 MB 00:00 2024-09-17T11:35:38.519 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:38.520 INFO:teuthology.orchestra.run.smithi049.stdout:Total 9.0 MB/s | 3.2 MB 00:00 2024-09-17T11:35:38.520 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:38.531 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:38.531 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:38.617 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:38.618 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:38.739 DEBUG:teuthology.parallel:result is None 2024-09-17T11:35:39.085 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T11:35:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:40.297 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:40.297 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:40.539 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-nbd 2024-09-17T11:35:41.153 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 11:33:49 AM UTC. 2024-09-17T11:35:41.291 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-09-17T11:35:41.292 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:41.292 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-09-17T11:35:41.292 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:41.292 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.3.0-4937.g74984932.el9 ceph 171 k 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 171 k 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 490 k 2024-09-17T11:35:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-09-17T11:35:41.498 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-nbd-19.3.0-4937.g74984932.el9.x86_64.rpm 837 kB/s | 171 kB 00:00 2024-09-17T11:35:41.498 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:35:41.498 INFO:teuthology.orchestra.run.smithi049.stdout:Total 830 kB/s | 171 kB 00:00 2024-09-17T11:35:41.499 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-09-17T11:35:41.507 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-09-17T11:35:41.507 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-09-17T11:35:41.576 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-09-17T11:35:41.577 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-09-17T11:35:41.741 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-09-17T11:35:41.840 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:42.264 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:42.558 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 1/1 2024-09-17T11:35:42.558 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:42.558 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-09-17T11:35:42.558 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.3.0-4937.g74984932.el9.x86_64 2024-09-17T11:35:42.559 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:35:42.559 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-09-17T11:35:42.795 DEBUG:teuthology.parallel:result is None 2024-09-17T11:35:42.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:35:42.935 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T11:35:42.957 INFO:teuthology.orchestra.run.smithi049.stdout:19.3.0-4937.g74984932.el9 2024-09-17T11:35:42.957 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4937.g74984932.el9 2024-09-17T11:35:42.958 INFO:teuthology.task.install:The correct ceph version 19.3.0-4937.g74984932 is installed. 2024-09-17T11:35:42.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:35:43.102 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T11:35:43.128 INFO:teuthology.orchestra.run.smithi129.stdout:19.3.0-4937.g74984932.el9 2024-09-17T11:35:43.128 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4937.g74984932.el9 2024-09-17T11:35:43.129 INFO:teuthology.task.install:The correct ceph version 19.3.0-4937.g74984932 is installed. 2024-09-17T11:35:43.130 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T11:35:43.131 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:35:43.131 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T11:35:43.157 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:35:43.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T11:35:43.199 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T11:35:43.199 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:35:43.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T11:35:43.224 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T11:35:43.290 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:35:43.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T11:35:43.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T11:35:43.384 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T11:35:43.385 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:35:43.385 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T11:35:43.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T11:35:43.491 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:35:43.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T11:35:43.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T11:35:43.584 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T11:35:43.584 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:35:43.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T11:35:43.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T11:35:43.676 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:35:43.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T11:35:43.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T11:35:43.769 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T11:35:43.876 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', 'CEPHADM_STRAY_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': '7498493279982dfe87f00616198a5967475f6169', 'cephadm_mode': 'cephadm-package'} 2024-09-17T11:35:43.876 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:35:43.877 INFO:tasks.cephadm:Cluster fsid is f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:35:43.877 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T11:35:43.877 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.c': '[v2:172.21.15.49:3301,v1:172.21.15.49:6790]', 'mon.b': '172.21.15.129'} 2024-09-17T11:35:43.877 INFO:tasks.cephadm:First mon is mon.a on smithi049 2024-09-17T11:35:43.877 INFO:tasks.cephadm:First mgr is y 2024-09-17T11:35:43.877 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T11:35:43.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-09-17T11:35:43.904 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-09-17T11:35:43.940 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 on all hosts... 2024-09-17T11:35:43.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 pull 2024-09-17T11:35:43.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 pull 2024-09-17T11:35:45.545 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169... 2024-09-17T11:35:45.598 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169... 2024-09-17T11:36:14.941 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-09-17T11:36:14.941 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev)", 2024-09-17T11:36:14.941 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76", 2024-09-17T11:36:14.941 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-09-17T11:36:14.942 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb" 2024-09-17T11:36:14.942 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-09-17T11:36:14.942 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-09-17T11:36:21.788 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev)", 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76", 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb" 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T11:36:21.789 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T11:36:21.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-09-17T11:36:21.847 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-09-17T11:36:21.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-09-17T11:36:21.917 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-09-17T11:36:21.942 INFO:tasks.cephadm:Writing seed config... 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T11:36:21.945 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-17T11:36:21.946 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T11:36:21.947 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:36:21.947 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T11:36:21.975 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f97d9e96-74e8-11ef-bceb-c7b262605968 mon election default strategy = 3 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-17T11:36:21.976 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T11:36:22.018 DEBUG:teuthology.orchestra.run.smithi049:mgr.y> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T11:36:22.062 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T11:36:22.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 -v bootstrap --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T11:36:22.275 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-09-17T11:36:22.275 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169', '-v', 'bootstrap', '--fsid', 'f97d9e96-74e8-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-09-17T11:36:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T11:36:22.275 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-09-17T11:36:22.300 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 5.2.2 2024-09-17T11:36:22.300 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-09-17T11:36:22.300 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-09-17T11:36:22.307 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T11:36:22.308 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T11:36:22.315 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T11:36:22.316 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:36:22.321 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-09-17T11:36:22.328 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-09-17T11:36:22.328 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-09-17T11:36:22.328 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-09-17T11:36:22.350 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 5.2.2 2024-09-17T11:36:22.350 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T11:36:22.350 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-09-17T11:36:22.351 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-09-17T11:36:22.357 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T11:36:22.357 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T11:36:22.363 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T11:36:22.364 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:36:22.370 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-09-17T11:36:22.377 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-09-17T11:36:22.377 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-09-17T11:36:22.377 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 140459285780848 on /run/cephadm/f97d9e96-74e8-11ef-bceb-c7b262605968.lock 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 140459285780848 acquired on /run/cephadm/f97d9e96-74e8-11ef-bceb-c7b262605968.lock 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-09-17T11:36:22.378 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-09-17T11:36:22.381 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-09-17T11:36:22.382 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-09-17T11:36:22.384 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T11:36:22.384 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-17T11:36:22.384 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link 2024-09-17T11:36:22.387 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T11:36:22.388 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-09-17T11:36:22.388 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-09-17T11:36:22.388 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T11:36:22.388 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T11:36:22.389 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169... 2024-09-17T11:36:22.892 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169... 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:89cd78ca29aaec9efd0f7c5ff71447705f964d133563bccc90c7979c6d4b526f 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 2024-09-17T11:36:22.893 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T11:36:23.797 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev) 2024-09-17T11:36:23.797 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev) 2024-09-17T11:36:23.797 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T11:36:24.812 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-09-17T11:36:24.812 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-09-17T11:36:25.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQC5aelm5VO4CRAAVEyz1CcSfOYVb676j5d5uw== 2024-09-17T11:36:30.437 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQC6aelmd/+RChAArKA14TgihE/VtqTYpjsIOA== 2024-09-17T11:36:39.789 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDBaelmXfBjGhAA5x1c0CMXvZCDkTKX6p1+2Q== 2024-09-17T11:36:39.789 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-09-17T11:36:40.694 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T11:36:40.694 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-17T11:36:40.694 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:40.694 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T11:36:40.694 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T11:36:40.695 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-09-17T11:36:40.695 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:40.695 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T11:36:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:36:40.695 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-09-17T11:36:41.633 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2024-09-17T11:36:41.845 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-17T11:36:41.991 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T11:36:42.168 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target. 2024-09-17T11:36:42.168 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target. 2024-09-17T11:36:42.406 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a 2024-09-17T11:36:42.406 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T11:36:42.575 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target.wants/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@.service. 2024-09-17T11:36:43.093 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:43 smithi049 ceph-mon[29539]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T11:36:43.107 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T11:36:43.107 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-09-17T11:36:43.114 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T11:36:43.114 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:36:43.114 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-09-17T11:36:43.114 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T11:36:43.115 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-09-17T11:36:43.115 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-09-17T11:36:43.343 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:43 smithi049 bash[29501]: 4cc4910cd9f7b628d24ac48b6eca69eb987a98027560299883621d9f297c32cf 2024-09-17T11:36:43.343 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:43 smithi049 systemd[1]: Started Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: monmap epoch 1 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: last_changed 2024-09-17T11:36:40.153935+0000 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: min_mon_release 19 (squid) 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: election_strategy: 1 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: fsmap 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T11:36:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: mgrmap e1: no daemons active 2024-09-17T11:36:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:44 smithi049 ceph-mon[29539]: from='client.? 172.21.15.49:0/3858488823' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558866s) 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-09-17T11:36:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.543+0000 7f5e7bf13640 1 Processor -- start 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.543+0000 7f5e7bf13640 1 -- start start 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e7bf13640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e7bf13640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74107350 con 0x7f5e74106960 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e79caf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e79caf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35250/0 (socket says 172.21.15.49:35250) 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e79caf640 1 -- 172.21.15.49:0/2107509916 learned_addr learned my addr 172.21.15.49:0/2107509916 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.544+0000 7f5e79caf640 1 -- 172.21.15.49:0/2107509916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e741074d0 con 0x7f5e74106960 2024-09-17T11:36:44.193 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.546+0000 7f5e79caf640 1 --2- 172.21.15.49:0/2107509916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009820 tx=0x7f5e7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c5789299c97f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.546+0000 7f5e78cad640 1 -- 172.21.15.49:0/2107509916 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e70037070 con 0x7f5e74106960 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.546+0000 7f5e78cad640 1 -- 172.21.15.49:0/2107509916 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e70032440 con 0x7f5e74106960 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.546+0000 7f5e78cad640 1 -- 172.21.15.49:0/2107509916 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e70032740 con 0x7f5e74106960 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- 172.21.15.49:0/2107509916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 msgr2=0x7f5e74106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 --2- 172.21.15.49:0/2107509916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009820 tx=0x7f5e7002a270 comp rx=0 tx=0).stop 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- 172.21.15.49:0/2107509916 shutdown_connections 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 --2- 172.21.15.49:0/2107509916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e74106d60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- 172.21.15.49:0/2107509916 >> 172.21.15.49:0/2107509916 conn(0x7f5e74102250 msgr2=0x7f5e74104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- 172.21.15.49:0/2107509916 shutdown_connections 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- 172.21.15.49:0/2107509916 wait complete. 2024-09-17T11:36:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 Processor -- start 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.547+0000 7f5e7bf13640 1 -- start start 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e7bf13640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e7bf13640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74107e70 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e79caf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e79caf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35260/0 (socket says 172.21.15.49:35260) 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e79caf640 1 -- 172.21.15.49:0/3858488823 learned_addr learned my addr 172.21.15.49:0/3858488823 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e79caf640 1 -- 172.21.15.49:0/3858488823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7419a6d0 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e79caf640 1 --2- 172.21.15.49:0/3858488823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e70041070 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7003c7a0 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7419a960 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.548+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e70037040 con 0x7f5e74106960 2024-09-17T11:36:44.195 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.549+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7419b320 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.549+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e70044070 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.549+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e7003d3f0 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.549+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e74108620 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.552+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5e70046450 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.616+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5e7419c060 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.616+0000 7f5e6affd640 1 -- 172.21.15.49:0/3858488823 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f5e70046630 con 0x7f5e74106960 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 msgr2=0x7f5e7419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 --2- 172.21.15.49:0/3858488823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e700046c0 comp rx=0 tx=0).stop 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 shutdown_connections 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 --2- 172.21.15.49:0/3858488823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e74106960 0x7f5e7419a170 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 >> 172.21.15.49:0/3858488823 conn(0x7f5e74102250 msgr2=0x7f5e74190c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:44.196 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 shutdown_connections 2024-09-17T11:36:44.197 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:43.617+0000 7f5e7bf13640 1 -- 172.21.15.49:0/3858488823 wait complete. 2024-09-17T11:36:44.197 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-09-17T11:36:44.197 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T11:36:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:45 smithi049 ceph-mon[29539]: from='client.? 172.21.15.49:0/2728125713' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-17T11:36:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:45 smithi049 ceph-mon[29539]: from='client.? 172.21.15.49:0/2728125713' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T11:36:46.444 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:46.444 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.641+0000 7f73fbd44640 1 Processor -- start 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.641+0000 7f73fbd44640 1 -- start start 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73fbd44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73fbd44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4106250 con 0x7f73f4105860 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73f9ae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73f9ae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35264/0 (socket says 172.21.15.49:35264) 2024-09-17T11:36:46.446 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73f9ae0640 1 -- 172.21.15.49:0/2245214086 learned_addr learned my addr 172.21.15.49:0/2245214086 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.642+0000 7f73f9ae0640 1 -- 172.21.15.49:0/2245214086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f41063d0 con 0x7f73f4105860 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73f9ae0640 1 --2- 172.21.15.49:0/2245214086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009820 tx=0x7f73f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed736d36b446483 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73f8ade640 1 -- 172.21.15.49:0/2245214086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73f0037070 con 0x7f73f4105860 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73f8ade640 1 -- 172.21.15.49:0/2245214086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f0032440 con 0x7f73f4105860 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73f8ade640 1 -- 172.21.15.49:0/2245214086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73f0032740 con 0x7f73f4105860 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73fbd44640 1 -- 172.21.15.49:0/2245214086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 msgr2=0x7f73f4105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73fbd44640 1 --2- 172.21.15.49:0/2245214086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009820 tx=0x7f73f002a270 comp rx=0 tx=0).stop 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73fbd44640 1 -- 172.21.15.49:0/2245214086 shutdown_connections 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.643+0000 7f73fbd44640 1 --2- 172.21.15.49:0/2245214086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f4105c60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 -- 172.21.15.49:0/2245214086 >> 172.21.15.49:0/2245214086 conn(0x7f73f41011b0 msgr2=0x7f73f41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 -- 172.21.15.49:0/2245214086 shutdown_connections 2024-09-17T11:36:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 -- 172.21.15.49:0/2245214086 wait complete. 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 Processor -- start 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 -- start start 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73fbd44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4106d70 con 0x7f73f4105860 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73f9ae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73f9ae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35266/0 (socket says 172.21.15.49:35266) 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.644+0000 7f73f9ae0640 1 -- 172.21.15.49:0/2728125713 learned_addr learned my addr 172.21.15.49:0/2728125713 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73f9ae0640 1 -- 172.21.15.49:0/2728125713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f419a6d0 con 0x7f73f4105860 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73f9ae0640 1 --2- 172.21.15.49:0/2728125713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f73f00090b0 tx=0x7f73f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73f0041070 con 0x7f73f4105860 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f003c7a0 con 0x7f73f4105860 2024-09-17T11:36:46.448 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f419a960 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73f0037040 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f419b320 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f73f0044070 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.645+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f73f003d3f0 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.646+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f4107520 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.648+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f73f0046450 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.710+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f73f419c0c0 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.712+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+374 (secure 0 0 0) 0x7f73f0046630 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.713+0000 7f73eaffd640 1 -- 172.21.15.49:0/2728125713 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f73f0056070 con 0x7f73f4105860 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 msgr2=0x7f73f419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 --2- 172.21.15.49:0/2728125713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f73f00090b0 tx=0x7f73f00046c0 comp rx=0 tx=0).stop 2024-09-17T11:36:46.449 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 shutdown_connections 2024-09-17T11:36:46.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 --2- 172.21.15.49:0/2728125713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73f4105860 0x7f73f419a170 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:46.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 >> 172.21.15.49:0/2728125713 conn(0x7f73f41011b0 msgr2=0x7f73f4190c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:46.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 shutdown_connections 2024-09-17T11:36:46.450 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:44.714+0000 7f73fbd44640 1 -- 172.21.15.49:0/2728125713 wait complete. 2024-09-17T11:36:46.450 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-09-17T11:36:47.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:46 smithi049 ceph-mon[29539]: from='client.? 172.21.15.49:0/3621340567' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:36:47.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.877+0000 7ff32b79f640 1 Processor -- start 2024-09-17T11:36:47.499 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32b79f640 1 -- start start 2024-09-17T11:36:47.499 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32b79f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:47.499 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32b79f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff324109a60 con 0x7ff32407a440 2024-09-17T11:36:47.499 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35278/0 (socket says 172.21.15.49:35278) 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32953b640 1 -- 172.21.15.49:0/1419865435 learned_addr learned my addr 172.21.15.49:0/1419865435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.878+0000 7ff32953b640 1 -- 172.21.15.49:0/1419865435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff324109be0 con 0x7ff32407a440 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.879+0000 7ff32953b640 1 --2- 172.21.15.49:0/1419865435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff320009820 tx=0x7ff32002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e74ca26f139942 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.879+0000 7ff31bfff640 1 -- 172.21.15.49:0/1419865435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff320037070 con 0x7ff32407a440 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.879+0000 7ff31bfff640 1 -- 172.21.15.49:0/1419865435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7ff3200328a0 con 0x7ff32407a440 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.879+0000 7ff31bfff640 1 -- 172.21.15.49:0/1419865435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff320030710 con 0x7ff32407a440 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 -- 172.21.15.49:0/1419865435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 msgr2=0x7ff324109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 --2- 172.21.15.49:0/1419865435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff320009820 tx=0x7ff32002a300 comp rx=0 tx=0).stop 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 -- 172.21.15.49:0/1419865435 shutdown_connections 2024-09-17T11:36:47.500 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 --2- 172.21.15.49:0/1419865435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff324109520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 -- 172.21.15.49:0/1419865435 >> 172.21.15.49:0/1419865435 conn(0x7ff3241011b0 msgr2=0x7ff3241035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 -- 172.21.15.49:0/1419865435 shutdown_connections 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.880+0000 7ff32b79f640 1 -- 172.21.15.49:0/1419865435 wait complete. 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32b79f640 1 Processor -- start 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32b79f640 1 -- start start 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32b79f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32b79f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff32410aca0 con 0x7ff32407a440 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32953b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35286/0 (socket says 172.21.15.49:35286) 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32953b640 1 -- 172.21.15.49:0/3621340567 learned_addr learned my addr 172.21.15.49:0/3621340567 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.881+0000 7ff32953b640 1 -- 172.21.15.49:0/3621340567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32419ae20 con 0x7ff32407a440 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff32953b640 1 --2- 172.21.15.49:0/3621340567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff320030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff320037040 con 0x7ff32407a440 2024-09-17T11:36:47.501 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7ff320040070 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32003bdf0 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32419b0b0 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32419ba70 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff320045460 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.882+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff320044730 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.883+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff324107590 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.885+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff32003b630 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.947+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff32419bcd0 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.947+0000 7ff31a7fc640 1 -- 172.21.15.49:0/3621340567 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff32003b810 con 0x7ff32407a440 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 msgr2=0x7ff32419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 --2- 172.21.15.49:0/3621340567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff320030800 comp rx=0 tx=0).stop 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 shutdown_connections 2024-09-17T11:36:47.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 --2- 172.21.15.49:0/3621340567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff32407a440 0x7ff32419a8c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:47.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 >> 172.21.15.49:0/3621340567 conn(0x7ff3241011b0 msgr2=0x7ff324101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:47.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 shutdown_connections 2024-09-17T11:36:47.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:46.948+0000 7ff32b79f640 1 -- 172.21.15.49:0/3621340567 wait complete. 2024-09-17T11:36:47.504 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-09-17T11:36:47.789 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:47 smithi049 systemd[1]: Stopping Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:36:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:47 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a[29516]: 2024-09-17T11:36:47.787+0000 7fe1236d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T11:36:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:47 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a[29516]: 2024-09-17T11:36:47.787+0000 7fe1236d1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T11:36:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:47 smithi049 podman[29869]: 2024-09-17 11:36:47.951621464 +0000 UTC m=+0.285510300 container died 4cc4910cd9f7b628d24ac48b6eca69eb987a98027560299883621d9f297c32cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-17T11:36:48.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 podman[29869]: 2024-09-17 11:36:48.302641891 +0000 UTC m=+0.636530714 container cleanup 4cc4910cd9f7b628d24ac48b6eca69eb987a98027560299883621d9f297c32cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-7498493, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-17T11:36:48.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 bash[29869]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a 2024-09-17T11:36:48.488 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 podman[29885]: 2024-09-17 11:36:48.486795767 +0000 UTC m=+0.537998337 container remove 4cc4910cd9f7b628d24ac48b6eca69eb987a98027560299883621d9f297c32cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-17T11:36:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T11:36:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 systemd[1]: Stopped Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:36:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 systemd[1]: Starting Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 podman[29978]: 2024-09-17 11:36:48.834330204 +0000 UTC m=+0.016190945 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:48 smithi049 podman[29978]: 2024-09-17 11:36:48.939179945 +0000 UTC m=+0.121040711 container create b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 podman[29978]: 2024-09-17 11:36:49.131362185 +0000 UTC m=+0.313222922 container init b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-7498493, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 podman[29978]: 2024-09-17 11:36:49.136291827 +0000 UTC m=+0.318152553 container start b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-7498493) 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev), process ceph-mon, pid 7 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: pidfile_write: ignore empty --pid-file 2024-09-17T11:36:49.199 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: load: jerasure load: lrc 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: RocksDB version: 7.9.2 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Git sha 0 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Compile date 2024-09-15 20:52:36 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: DB SUMMARY 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: DB Session ID: G5OFVMMRVB5ZYX4M57KL 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: CURRENT file: CURRENT 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 91892 ; 2024-09-17T11:36:49.200 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.error_if_exists: 0 2024-09-17T11:36:49.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.create_if_missing: 0 2024-09-17T11:36:49.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.paranoid_checks: 1 2024-09-17T11:36:49.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T11:36:49.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T11:36:49.201 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.env: 0x561cf691f120 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.info_log: 0x561cf87cf700 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.statistics: (nil) 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.use_fsync: 0 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_log_file_size: 0 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_fallocate: 1 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T11:36:49.202 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.use_direct_reads: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.db_log_dir: 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.wal_dir: 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T11:36:49.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.write_buffer_manager: 0x561cf87d3b80 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.unordered_write: 0 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T11:36:49.204 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.row_cache: None 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.wal_filter: None 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.two_write_queues: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.wal_compression: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.atomic_flush: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.log_readahead_size: 0 2024-09-17T11:36:49.205 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_background_jobs: 2 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_background_compactions: -1 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_subcompactions: 1 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T11:36:49.206 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_open_files: -1 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_background_flushes: -1 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Compression algorithms supported: 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kZSTD supported: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kXpressCompression supported: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kBZip2Compression supported: 0 2024-09-17T11:36:49.207 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kLZ4Compression supported: 1 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kZlibCompression supported: 1 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: kSnappyCompression supported: 1 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.merge_operator: 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_filter: None 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T11:36:49.208 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561cf87cf320) 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2024-09-17T11:36:49.209 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x561cf87c3610 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2024-09-17T11:36:49.210 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T11:36:49.211 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression: NoCompression 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.num_levels: 7 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:36:49.212 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:36:49.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T11:36:49.214 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T11:36:49.214 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T11:36:49.214 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T11:36:49.214 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T11:36:49.214 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T11:36:49.215 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T11:36:49.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.inplace_update_support: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.bloom_locality: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.max_successive_merges: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T11:36:49.217 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.ttl: 2592000 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enable_blob_files: false 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.min_blob_size: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T11:36:49.218 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 229e18f8-21a8-47ee-a137-08e0c0b0bfc3 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573009162790, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573009163768, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 88321, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 273, "table_properties": {"data_size": 86389, "index_size": 252, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 10932, "raw_average_key_size": 47, "raw_value_size": 80207, "raw_average_value_size": 345, "num_data_blocks": 11, "num_entries": 232, "num_filter_entries": 232, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726573009, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "229e18f8-21a8-47ee-a137-08e0c0b0bfc3", "db_session_id": "G5OFVMMRVB5ZYX4M57KL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573009163841, "job": 1, "event": "recovery_finished"} 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561cf87f2e00 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: DB pointer 0x561cf88f4000 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T11:36:49.219 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 88.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 88.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T11:36:49.220 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T11:36:49.221 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 27.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 27.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x561cf87c3610#7 capacity: 512.00 MB usage: 1.27 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???) e1 preinit fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).mds e1 new map 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).mds e1 print_map 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: btime 2024-09-17T11:36:43:058780+0000 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T11:36:49.222 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 ceph-mon[30017]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 bash[29978]: b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b 2024-09-17T11:36:49.223 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:49 smithi049 systemd[1]: Started Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:36:49.223 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T11:36:50.280 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T11:36:50.280 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: monmap epoch 1 2024-09-17T11:36:50.280 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: last_changed 2024-09-17T11:36:40.153935+0000 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: min_mon_release 19 (squid) 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: election_strategy: 1 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: fsmap 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: mgrmap e1: no daemons active 2024-09-17T11:36:50.281 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2483121478' entity='client.admin' 2024-09-17T11:36:50.282 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.602+0000 7f30fad31640 1 Processor -- start 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30fad31640 1 -- start start 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30fad31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30fad31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30f407c950 con 0x7f30f407c4a0 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30f8acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30f8acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35300/0 (socket says 172.21.15.49:35300) 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30f8acd640 1 -- 172.21.15.49:0/1422655864 learned_addr learned my addr 172.21.15.49:0/1422655864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.603+0000 7f30f8acd640 1 -- 172.21.15.49:0/1422655864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f407b040 con 0x7f30f407c4a0 2024-09-17T11:36:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.605+0000 7f30f8acd640 1 --2- 172.21.15.49:0/1422655864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009820 tx=0x7f30e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa851d46888191c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.605+0000 7f30f37fe640 1 -- 172.21.15.49:0/1422655864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e8037070 con 0x7f30f407c4a0 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.605+0000 7f30f37fe640 1 -- 172.21.15.49:0/1422655864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f30e80328a0 con 0x7f30f407c4a0 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.605+0000 7f30f37fe640 1 -- 172.21.15.49:0/1422655864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e8030690 con 0x7f30f407c4a0 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 -- 172.21.15.49:0/1422655864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 msgr2=0x7f30f407ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 --2- 172.21.15.49:0/1422655864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009820 tx=0x7f30e802a270 comp rx=0 tx=0).stop 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 -- 172.21.15.49:0/1422655864 shutdown_connections 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 --2- 172.21.15.49:0/1422655864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f407ab00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 -- 172.21.15.49:0/1422655864 >> 172.21.15.49:0/1422655864 conn(0x7f30f41022d0 msgr2=0x7f30f41046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 -- 172.21.15.49:0/1422655864 shutdown_connections 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.606+0000 7f30fad31640 1 -- 172.21.15.49:0/1422655864 wait complete. 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30fad31640 1 Processor -- start 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30fad31640 1 -- start start 2024-09-17T11:36:50.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30fad31640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:50.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30fad31640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30f410cef0 con 0x7f30f407c4a0 2024-09-17T11:36:50.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30f8acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:50.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30f8acd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35314/0 (socket says 172.21.15.49:35314) 2024-09-17T11:36:50.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30f8acd640 1 -- 172.21.15.49:0/2483121478 learned_addr learned my addr 172.21.15.49:0/2483121478 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30f8acd640 1 -- 172.21.15.49:0/2483121478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f419d020 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.607+0000 7f30f8acd640 1 --2- 172.21.15.49:0/2483121478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f30e8006f10 tx=0x7f30e8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e8037040 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f30e8040070 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f419d2b0 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e803a390 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f419dc70 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f30e803a600 con 0x7f30f407c4a0 2024-09-17T11:36:50.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.608+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f30e8045770 con 0x7f30f407c4a0 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.609+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f407ab80 con 0x7f30f407c4a0 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.611+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f30e8032830 con 0x7f30f407c4a0 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.672+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f30f419e2f0 con 0x7f30f407c4a0 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.674+0000 7f30f1ffb640 1 -- 172.21.15.49:0/2483121478 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f30e803b430 con 0x7f30f407c4a0 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 msgr2=0x7f30f419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 --2- 172.21.15.49:0/2483121478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f30e8006f10 tx=0x7f30e8032670 comp rx=0 tx=0).stop 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 shutdown_connections 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 --2- 172.21.15.49:0/2483121478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f407c4a0 0x7f30f419cac0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 >> 172.21.15.49:0/2483121478 conn(0x7f30f41022d0 msgr2=0x7f30f4102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 shutdown_connections 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:49.675+0000 7f30fad31640 1 -- 172.21.15.49:0/2483121478 wait complete. 2024-09-17T11:36:50.290 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T11:36:50.313 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T11:36:50.313 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-09-17T11:36:50.314 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T11:36:50.314 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T11:36:50.572 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y 2024-09-17T11:36:50.572 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T11:36:50.731 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968.target.wants/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@.service. 2024-09-17T11:36:50.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:50 smithi049 systemd[1]: Starting Ceph mgr.y for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:36:51.309 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:50 smithi049 podman[30253]: 2024-09-17 11:36:50.86418286 +0000 UTC m=+0.016101995 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 2024-09-17T11:36:51.309 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:50 smithi049 podman[30253]: 2024-09-17 11:36:50.989952362 +0000 UTC m=+0.141871529 container create d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-7498493, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T11:36:51.310 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:51 smithi049 podman[30253]: 2024-09-17 11:36:51.200570422 +0000 UTC m=+0.352489543 container init d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=main-7498493, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-17T11:36:51.310 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:51 smithi049 podman[30253]: 2024-09-17 11:36:51.205576894 +0000 UTC m=+0.357496016 container start d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=main-7498493, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T11:36:51.310 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:51 smithi049 bash[30253]: d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 2024-09-17T11:36:51.323 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T11:36:51.323 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-09-17T11:36:51.331 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T11:36:51.331 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:36:51.331 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-09-17T11:36:51.331 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T11:36:51.340 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T11:36:51.340 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-09-17T11:36:51.348 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T11:36:51.348 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:36:51.348 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-09-17T11:36:51.348 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T11:36:51.349 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-09-17T11:36:51.349 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-09-17T11:36:51.592 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:36:51 smithi049 systemd[1]: Started Ceph mgr.y for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:36:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/379777499' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-17T11:36:52.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "f97d9e96-74e8-11ef-bceb-c7b262605968", 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T11:36:52.709 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T11:36:52.710 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T11:36:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T11:36:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T11:36:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T11:36:52.719 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "btime": "2024-09-17T11:36:43:058780+0000", 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T11:36:52.720 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T11:36:43.059430+0000", 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:52.721 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.972+0000 7f101257f640 1 Processor -- start 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.972+0000 7f101257f640 1 -- start start 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.972+0000 7f101257f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.972+0000 7f101257f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f100c109b40 con 0x7f100c07a360 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.973+0000 7f100bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.973+0000 7f100bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35342/0 (socket says 172.21.15.49:35342) 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.973+0000 7f100bfff640 1 -- 172.21.15.49:0/1877415620 learned_addr learned my addr 172.21.15.49:0/1877415620 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.973+0000 7f100bfff640 1 -- 172.21.15.49:0/1877415620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c109cc0 con 0x7f100c07a360 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.974+0000 7f100bfff640 1 --2- 172.21.15.49:0/1877415620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009820 tx=0x7f0ffc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbbcfe7e7b09283 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.974+0000 7f100affd640 1 -- 172.21.15.49:0/1877415620 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ffc037070 con 0x7f100c07a360 2024-09-17T11:36:52.722 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.974+0000 7f100affd640 1 -- 172.21.15.49:0/1877415620 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f0ffc0328a0 con 0x7f100c07a360 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.974+0000 7f100affd640 1 -- 172.21.15.49:0/1877415620 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ffc030690 con 0x7f100c07a360 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 -- 172.21.15.49:0/1877415620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 msgr2=0x7f100c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 --2- 172.21.15.49:0/1877415620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009820 tx=0x7f0ffc02a270 comp rx=0 tx=0).stop 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 -- 172.21.15.49:0/1877415620 shutdown_connections 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 --2- 172.21.15.49:0/1877415620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c109600 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 -- 172.21.15.49:0/1877415620 >> 172.21.15.49:0/1877415620 conn(0x7f100c1011b0 msgr2=0x7f100c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 -- 172.21.15.49:0/1877415620 shutdown_connections 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 -- 172.21.15.49:0/1877415620 wait complete. 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.975+0000 7f101257f640 1 Processor -- start 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f101257f640 1 -- start start 2024-09-17T11:36:52.723 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f101257f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f101257f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f100c10acc0 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f100bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f100bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35350/0 (socket says 172.21.15.49:35350) 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f100bfff640 1 -- 172.21.15.49:0/379777499 learned_addr learned my addr 172.21.15.49:0/379777499 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f100bfff640 1 -- 172.21.15.49:0/379777499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c19d010 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.976+0000 7f100bfff640 1 --2- 172.21.15.49:0/379777499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ffc037040 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f0ffc040070 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c19d2a0 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c19dc60 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ffc03be40 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ffc0454a0 con 0x7f100c07a360 2024-09-17T11:36:52.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.977+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ffc03a770 con 0x7f100c07a360 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.978+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff80050b0 con 0x7f100c07a360 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:51.980+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0ffc03b430 con 0x7f100c07a360 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.043+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ff8005670 con 0x7f100c07a360 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.044+0000 7f10097fa640 1 -- 172.21.15.49:0/379777499 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0ffc03b680 con 0x7f100c07a360 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 msgr2=0x7f100c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 --2- 172.21.15.49:0/379777499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc032740 comp rx=0 tx=0).stop 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 shutdown_connections 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 --2- 172.21.15.49:0/379777499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f100c07a360 0x7f100c19cab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 >> 172.21.15.49:0/379777499 conn(0x7f100c1011b0 msgr2=0x7f100c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 shutdown_connections 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:52.045+0000 7f101257f640 1 -- 172.21.15.49:0/379777499 wait complete. 2024-09-17T11:36:52.725 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-09-17T11:36:55.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2437393877' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-17T11:36:55.863 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:55.863 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:36:55.863 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "f97d9e96-74e8-11ef-bceb-c7b262605968", 2024-09-17T11:36:55.863 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T11:36:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T11:36:55.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T11:36:55.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T11:36:55.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T11:36:55.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "btime": "2024-09-17T11:36:43:058780+0000", 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T11:36:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T11:36:43.059430+0000", 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.166+0000 7f7c96e90640 1 Processor -- start 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.166+0000 7f7c96e90640 1 -- start start 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c96e90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c96e90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9007a6b0 con 0x7f7c9007a2b0 2024-09-17T11:36:55.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c94c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c94c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50702/0 (socket says 172.21.15.49:50702) 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c94c2c640 1 -- 172.21.15.49:0/1135575565 learned_addr learned my addr 172.21.15.49:0/1135575565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.167+0000 7f7c94c2c640 1 -- 172.21.15.49:0/1135575565 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9010bee0 con 0x7f7c9007a2b0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.168+0000 7f7c94c2c640 1 --2- 172.21.15.49:0/1135575565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009820 tx=0x7f7c8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcce076e8161942 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.168+0000 7f7c8f7fe640 1 -- 172.21.15.49:0/1135575565 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84037070 con 0x7f7c9007a2b0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.168+0000 7f7c8f7fe640 1 -- 172.21.15.49:0/1135575565 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f7c840328a0 con 0x7f7c9007a2b0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.168+0000 7f7c8f7fe640 1 -- 172.21.15.49:0/1135575565 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84030690 con 0x7f7c9007a2b0 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- 172.21.15.49:0/1135575565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 msgr2=0x7f7c9010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 --2- 172.21.15.49:0/1135575565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009820 tx=0x7f7c8402a270 comp rx=0 tx=0).stop 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- 172.21.15.49:0/1135575565 shutdown_connections 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 --2- 172.21.15.49:0/1135575565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9010b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:55.876 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- 172.21.15.49:0/1135575565 >> 172.21.15.49:0/1135575565 conn(0x7f7c901011b0 msgr2=0x7f7c901035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- 172.21.15.49:0/1135575565 shutdown_connections 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- 172.21.15.49:0/1135575565 wait complete. 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 Processor -- start 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- start start 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.169+0000 7f7c96e90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9010ce70 con 0x7f7c9007a2b0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c94c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c94c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50712/0 (socket says 172.21.15.49:50712) 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c94c2c640 1 -- 172.21.15.49:0/2437393877 learned_addr learned my addr 172.21.15.49:0/2437393877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c94c2c640 1 -- 172.21.15.49:0/2437393877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9019cf90 con 0x7f7c9007a2b0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c94c2c640 1 --2- 172.21.15.49:0/2437393877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006f10 tx=0x7f7c84032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c84037040 con 0x7f7c9007a2b0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f7c84040070 con 0x7f7c9007a2b0 2024-09-17T11:36:55.877 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c8403a390 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.170+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9019d220 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.171+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9019dc60 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.171+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c8403a600 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.171+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c84045770 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.171+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9010ba20 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.173+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7c84032830 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.237+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7c9019e210 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.238+0000 7f7c8dffb640 1 -- 172.21.15.49:0/2437393877 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7c8403b430 con 0x7f7c9007a2b0 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 msgr2=0x7f7c9019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 --2- 172.21.15.49:0/2437393877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006f10 tx=0x7f7c84032670 comp rx=0 tx=0).stop 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 shutdown_connections 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 --2- 172.21.15.49:0/2437393877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c9007a2b0 0x7f7c9019ca30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:55.878 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 >> 172.21.15.49:0/2437393877 conn(0x7f7c901011b0 msgr2=0x7f7c90101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:55.879 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 shutdown_connections 2024-09-17T11:36:55.879 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:55.239+0000 7f7c96e90640 1 -- 172.21.15.49:0/2437393877 wait complete. 2024-09-17T11:36:55.879 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: Activating manager daemon y 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: mgrmap e2: y(active, starting, since 0.00310654s) 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:36:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: Manager daemon y is now available 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' 2024-09-17T11:36:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:56 smithi049 ceph-mon[30017]: from='mgr.14100 172.21.15.49:0/3295650348' entity='mgr.y' 2024-09-17T11:36:58.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:58 smithi049 ceph-mon[30017]: mgrmap e3: y(active, since 1.00741s) 2024-09-17T11:36:59.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:36:59.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:36:59.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "f97d9e96-74e8-11ef-bceb-c7b262605968", 2024-09-17T11:36:59.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T11:36:59.075 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-17T11:36:59.076 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T11:36:59.077 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:59.078 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "btime": "2024-09-17T11:36:43:058780+0000", 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:59.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T11:36:43.059430+0000", 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.259+0000 7f59d03e4640 1 Processor -- start 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.259+0000 7f59d03e4640 1 -- start start 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.259+0000 7f59d03e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.259+0000 7f59d03e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8107350 con 0x7f59c8106960 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.260+0000 7f59ce180640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:59.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.260+0000 7f59ce180640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50766/0 (socket says 172.21.15.49:50766) 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.260+0000 7f59ce180640 1 -- 172.21.15.49:0/3336973163 learned_addr learned my addr 172.21.15.49:0/3336973163 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.260+0000 7f59ce180640 1 -- 172.21.15.49:0/3336973163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81074d0 con 0x7f59c8106960 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59ce180640 1 --2- 172.21.15.49:0/3336973163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009820 tx=0x7f59c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fa3f06fe981a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59cd17e640 1 -- 172.21.15.49:0/3336973163 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4037070 con 0x7f59c8106960 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59cd17e640 1 -- 172.21.15.49:0/3336973163 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f59c40328a0 con 0x7f59c8106960 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59cd17e640 1 -- 172.21.15.49:0/3336973163 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4030690 con 0x7f59c8106960 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59d03e4640 1 -- 172.21.15.49:0/3336973163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 msgr2=0x7f59c8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.261+0000 7f59d03e4640 1 --2- 172.21.15.49:0/3336973163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009820 tx=0x7f59c402a270 comp rx=0 tx=0).stop 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- 172.21.15.49:0/3336973163 shutdown_connections 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 --2- 172.21.15.49:0/3336973163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c8106d60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- 172.21.15.49:0/3336973163 >> 172.21.15.49:0/3336973163 conn(0x7f59c8102250 msgr2=0x7f59c8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- 172.21.15.49:0/3336973163 shutdown_connections 2024-09-17T11:36:59.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- 172.21.15.49:0/3336973163 wait complete. 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 Processor -- start 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- start start 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.262+0000 7f59d03e4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810ade0 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59ce180640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59ce180640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50772/0 (socket says 172.21.15.49:50772) 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59ce180640 1 -- 172.21.15.49:0/1493697025 learned_addr learned my addr 172.21.15.49:0/1493697025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59ce180640 1 -- 172.21.15.49:0/1493697025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c807cef0 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59ce180640 1 --2- 172.21.15.49:0/1493697025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000c00 tx=0x7f59c4032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4037040 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f59c4040070 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c403be40 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c807b2e0 con 0x7f59c8106960 2024-09-17T11:36:59.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.263+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c807b860 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.264+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 3) ==== 51613+0+0 (secure 0 0 0) 0x7f59c403b680 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.264+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c8106de0 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.264+0000 7f59bf7fe640 1 --2- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 0x7f59b4041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.264+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59c4073980 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.264+0000 7f59cd97f640 1 --2- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 0x7f59b4041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.265+0000 7f59cd97f640 1 --2- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 0x7f59b4041a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098c0 tx=0x7f59b8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.269+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f59c4036170 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.461+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 4) ==== 51719+0+0 (secure 0 0 0) 0x7f59c4045460 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.528+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f59c807be20 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.529+0000 7f59bf7fe640 1 -- 172.21.15.49:0/1493697025 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f59c4041270 con 0x7f59c8106960 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.531+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 msgr2=0x7f59b4041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:59.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.531+0000 7f59d03e4640 1 --2- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 0x7f59b4041a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098c0 tx=0x7f59b8006db0 comp rx=0 tx=0).stop 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.531+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 msgr2=0x7f59c807c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.531+0000 7f59d03e4640 1 --2- 172.21.15.49:0/1493697025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000c00 tx=0x7f59c4032740 comp rx=0 tx=0).stop 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 shutdown_connections 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 --2- 172.21.15.49:0/1493697025 >> v2:172.21.15.49:6800/1073451895 conn(0x7f59b403f5c0 0x7f59b4041a80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 --2- 172.21.15.49:0/1493697025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59c8106960 0x7f59c807c990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 >> 172.21.15.49:0/1493697025 conn(0x7f59c8102250 msgr2=0x7f59c8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 shutdown_connections 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:58.532+0000 7f59d03e4640 1 -- 172.21.15.49:0/1493697025 wait complete. 2024-09-17T11:36:59.084 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-09-17T11:36:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:59 smithi049 ceph-mon[30017]: mgrmap e4: y(active, since 2s) 2024-09-17T11:36:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:36:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1493697025' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-17T11:37:00.229 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:37:00.229 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T11:37:00.229 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T11:37:00.230 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9cad3e2640 1 Processor -- start 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9cad3e2640 1 -- start start 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9cad3e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9cad3e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8107350 con 0x7f9ca8106960 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50782/0 (socket says 172.21.15.49:50782) 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9ca6ffd640 1 -- 172.21.15.49:0/2204197882 learned_addr learned my addr 172.21.15.49:0/2204197882 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.472+0000 7f9ca6ffd640 1 -- 172.21.15.49:0/2204197882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca81074d0 con 0x7f9ca8106960 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.473+0000 7f9ca6ffd640 1 --2- 172.21.15.49:0/2204197882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009820 tx=0x7f9c9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee98a9c19702dd4f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.473+0000 7f9ca5ffb640 1 -- 172.21.15.49:0/2204197882 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98037070 con 0x7f9ca8106960 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.473+0000 7f9ca5ffb640 1 -- 172.21.15.49:0/2204197882 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f9c980328a0 con 0x7f9ca8106960 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.473+0000 7f9ca5ffb640 1 -- 172.21.15.49:0/2204197882 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98030690 con 0x7f9ca8106960 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2204197882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 msgr2=0x7f9ca8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:00.231 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2204197882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009820 tx=0x7f9c9802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2204197882 shutdown_connections 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2204197882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca8106d60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2204197882 >> 172.21.15.49:0/2204197882 conn(0x7f9ca8102250 msgr2=0x7f9ca8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2204197882 shutdown_connections 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2204197882 wait complete. 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.474+0000 7f9cad3e2640 1 Processor -- start 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9cad3e2640 1 -- start start 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9cad3e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9cad3e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca810ade0 con 0x7f9ca8106960 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50790/0 (socket says 172.21.15.49:50790) 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9ca6ffd640 1 -- 172.21.15.49:0/2693302296 learned_addr learned my addr 172.21.15.49:0/2693302296 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9ca6ffd640 1 -- 172.21.15.49:0/2693302296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca819ae40 con 0x7f9ca8106960 2024-09-17T11:37:00.232 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9ca6ffd640 1 --2- 172.21.15.49:0/2693302296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006f10 tx=0x7f9c98032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98037040 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f9c98040070 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca819b0d0 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.475+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c9803a390 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.476+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca819ba90 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.476+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 51719+0+0 (secure 0 0 0) 0x7f9c9803a650 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.476+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca8106de0 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.477+0000 7f9c8ffff640 1 --2- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 0x7f9c90041b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.477+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c98073d30 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.477+0000 7f9ca67fc640 1 --2- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 0x7f9c90041b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.477+0000 7f9ca67fc640 1 --2- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 0x7f9c90041b40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098c0 tx=0x7f9c9c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.481+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9c9803a940 con 0x7f9ca8106960 2024-09-17T11:37:00.233 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.668+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9ca819c1b0 con 0x7f9ca8106960 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.669+0000 7f9c8ffff640 1 -- 172.21.15.49:0/2693302296 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+374 (secure 0 0 0) 0x7f9c98073a90 con 0x7f9ca8106960 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 msgr2=0x7f9c90041b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 0x7f9c90041b40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098c0 tx=0x7f9c9c006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 msgr2=0x7f9ca819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2693302296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006f10 tx=0x7f9c98032670 comp rx=0 tx=0).stop 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 shutdown_connections 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2693302296 >> v2:172.21.15.49:6800/1073451895 conn(0x7f9c9003f680 0x7f9c90041b40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 --2- 172.21.15.49:0/2693302296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ca8106960 0x7f9ca819a8e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 >> 172.21.15.49:0/2693302296 conn(0x7f9ca8102250 msgr2=0x7f9ca8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 shutdown_connections 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:36:59.672+0000 7f9cad3e2640 1 -- 172.21.15.49:0/2693302296 wait complete. 2024-09-17T11:37:00.234 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-09-17T11:37:00.607 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2693302296' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-17T11:37:01.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/32286682' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-17T11:37:01.785 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:01 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setuser ceph since I am not root 2024-09-17T11:37:01.785 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:01 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setgroup ceph since I am not root 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.621+0000 7f45d1cde640 1 Processor -- start 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45d1cde640 1 -- start start 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45d1cde640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45d1cde640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc07d190 con 0x7f45cc07cce0 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50800/0 (socket says 172.21.15.49:50800) 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45cb7fe640 1 -- 172.21.15.49:0/1184832399 learned_addr learned my addr 172.21.15.49:0/1184832399 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.622+0000 7f45cb7fe640 1 -- 172.21.15.49:0/1184832399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc07b880 con 0x7f45cc07cce0 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.623+0000 7f45cb7fe640 1 --2- 172.21.15.49:0/1184832399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009820 tx=0x7f45bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85acb096824a63aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.623+0000 7f45ca7fc640 1 -- 172.21.15.49:0/1184832399 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc037070 con 0x7f45cc07cce0 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.623+0000 7f45ca7fc640 1 -- 172.21.15.49:0/1184832399 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f45bc0328a0 con 0x7f45cc07cce0 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.623+0000 7f45ca7fc640 1 -- 172.21.15.49:0/1184832399 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc030690 con 0x7f45cc07cce0 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 -- 172.21.15.49:0/1184832399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 msgr2=0x7f45cc07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 --2- 172.21.15.49:0/1184832399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009820 tx=0x7f45bc02a270 comp rx=0 tx=0).stop 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 -- 172.21.15.49:0/1184832399 shutdown_connections 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 --2- 172.21.15.49:0/1184832399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc07b340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 -- 172.21.15.49:0/1184832399 >> 172.21.15.49:0/1184832399 conn(0x7f45cc102250 msgr2=0x7f45cc104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 -- 172.21.15.49:0/1184832399 shutdown_connections 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 -- 172.21.15.49:0/1184832399 wait complete. 2024-09-17T11:37:02.080 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.624+0000 7f45d1cde640 1 Processor -- start 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45d1cde640 1 -- start start 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45d1cde640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45d1cde640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc10ceb0 con 0x7f45cc07cce0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50804/0 (socket says 172.21.15.49:50804) 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45cb7fe640 1 -- 172.21.15.49:0/32286682 learned_addr learned my addr 172.21.15.49:0/32286682 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45cb7fe640 1 -- 172.21.15.49:0/32286682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc19d000 con 0x7f45cc07cce0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45cb7fe640 1 --2- 172.21.15.49:0/32286682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc037040 con 0x7f45cc07cce0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f45bc040070 con 0x7f45cc07cce0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc19d290 con 0x7f45cc07cce0 2024-09-17T11:37:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.625+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc03be40 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.626+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc19dc50 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 51719+0+0 (secure 0 0 0) 0x7f45bc045460 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc07b3c0 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45c8ff9640 1 --2- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 0x7f45b8041b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f45bc073c40 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45caffd640 1 --2- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 0x7f45b8041b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.627+0000 7f45caffd640 1 --2- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 0x7f45b8041b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098c0 tx=0x7f45b4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.632+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f45bc03b430 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:00.868+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f45cc19e370 con 0x7f45cc07cce0 2024-09-17T11:37:02.082 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.467+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f45bc0739a0 con 0x7f45cc07cce0 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.469+0000 7f45c8ff9640 1 -- 172.21.15.49:0/32286682 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 5) ==== 51730+0+0 (secure 0 0 0) 0x7f45bc045750 con 0x7f45cc07cce0 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 msgr2=0x7f45b8041b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 --2- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 0x7f45b8041b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098c0 tx=0x7f45b4006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 msgr2=0x7f45cc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 --2- 172.21.15.49:0/32286682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc032740 comp rx=0 tx=0).stop 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 shutdown_connections 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 --2- 172.21.15.49:0/32286682 >> v2:172.21.15.49:6800/1073451895 conn(0x7f45b803f6d0 0x7f45b8041b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 --2- 172.21.15.49:0/32286682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45cc07cce0 0x7f45cc19caa0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:02.083 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 >> 172.21.15.49:0/32286682 conn(0x7f45cc102250 msgr2=0x7f45cc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:02.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 shutdown_connections 2024-09-17T11:37:02.084 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:01.471+0000 7f45d1cde640 1 -- 172.21.15.49:0/32286682 wait complete. 2024-09-17T11:37:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/32286682' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T11:37:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:02 smithi049 ceph-mon[30017]: mgrmap e5: y(active, since 5s) 2024-09-17T11:37:03.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.536+0000 7fa496f47640 1 Processor -- start 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.536+0000 7fa496f47640 1 -- start start 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa496f47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa496f47640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa490109a60 con 0x7fa49007a440 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa494ce3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:03.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa494ce3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50840/0 (socket says 172.21.15.49:50840) 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa494ce3640 1 -- 172.21.15.49:0/3184798436 learned_addr learned my addr 172.21.15.49:0/3184798436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.537+0000 7fa494ce3640 1 -- 172.21.15.49:0/3184798436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa490109be0 con 0x7fa49007a440 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.538+0000 7fa494ce3640 1 --2- 172.21.15.49:0/3184798436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa484009820 tx=0x7fa48402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f428e7c183f623 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.538+0000 7fa48f7fe640 1 -- 172.21.15.49:0/3184798436 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484037070 con 0x7fa49007a440 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.538+0000 7fa48f7fe640 1 -- 172.21.15.49:0/3184798436 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fa4840328a0 con 0x7fa49007a440 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.538+0000 7fa48f7fe640 1 -- 172.21.15.49:0/3184798436 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484030710 con 0x7fa49007a440 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.538+0000 7fa496f47640 1 -- 172.21.15.49:0/3184798436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 msgr2=0x7fa490109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 --2- 172.21.15.49:0/3184798436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa484009820 tx=0x7fa48402a300 comp rx=0 tx=0).stop 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 -- 172.21.15.49:0/3184798436 shutdown_connections 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 --2- 172.21.15.49:0/3184798436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa490109520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 -- 172.21.15.49:0/3184798436 >> 172.21.15.49:0/3184798436 conn(0x7fa4901011b0 msgr2=0x7fa4901035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 -- 172.21.15.49:0/3184798436 shutdown_connections 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 -- 172.21.15.49:0/3184798436 wait complete. 2024-09-17T11:37:03.377 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 Processor -- start 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.539+0000 7fa496f47640 1 -- start start 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa496f47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa496f47640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa49010aca0 con 0x7fa49007a440 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa494ce3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa494ce3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50848/0 (socket says 172.21.15.49:50848) 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa494ce3640 1 -- 172.21.15.49:0/1678351673 learned_addr learned my addr 172.21.15.49:0/1678351673 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa494ce3640 1 -- 172.21.15.49:0/1678351673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49019ae20 con 0x7fa49007a440 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa494ce3640 1 --2- 172.21.15.49:0/1678351673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa484030460 tx=0x7fa484030710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484037040 con 0x7fa49007a440 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fa484040070 con 0x7fa49007a440 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49019b0b0 con 0x7fa49007a440 2024-09-17T11:37:03.378 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.540+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48403be30 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.541+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49019ba70 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.541+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa490107590 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.542+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 51730+0+0 (secure 0 0 0) 0x7fa484045460 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.542+0000 7fa48dffb640 1 --2- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 0x7fa480041b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.542+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa484073d90 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.542+0000 7fa48ffff640 1 -- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 msgr2=0x7fa480041b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.542+0000 7fa48ffff640 1 --2- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 0x7fa480041b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.550+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fa484079020 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.743+0000 7fa48ffff640 1 -- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 msgr2=0x7fa480041b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.743+0000 7fa48ffff640 1 --2- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 0x7fa480041b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.788+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa49019c190 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.788+0000 7fa48dffb640 1 -- 172.21.15.49:0/1678351673 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa484073af0 con 0x7fa49007a440 2024-09-17T11:37:03.379 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 msgr2=0x7fa480041b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 --2- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 0x7fa480041b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 msgr2=0x7fa49019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 --2- 172.21.15.49:0/1678351673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa484030460 tx=0x7fa484030710 comp rx=0 tx=0).stop 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 shutdown_connections 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 --2- 172.21.15.49:0/1678351673 >> v2:172.21.15.49:6800/1073451895 conn(0x7fa48003f6d0 0x7fa480041b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 --2- 172.21.15.49:0/1678351673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49007a440 0x7fa49019a8c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 >> 172.21.15.49:0/1678351673 conn(0x7fa4901011b0 msgr2=0x7fa490101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 shutdown_connections 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:02.791+0000 7fa496f47640 1 -- 172.21.15.49:0/1678351673 wait complete. 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-09-17T11:37:03.380 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-09-17T11:37:03.623 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1678351673' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-17T11:37:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: Active manager daemon y restarted 2024-09-17T11:37:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: Activating manager daemon y 2024-09-17T11:37:07.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T11:37:07.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: mgrmap e6: y(active, starting, since 0.00353354s) 2024-09-17T11:37:07.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:37:07.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:37:07.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:37:07.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:37:07.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:37:07.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:06 smithi049 ceph-mon[30017]: Manager daemon y is now available 2024-09-17T11:37:08.216 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.792+0000 7f25ebf87640 1 Processor -- start 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.792+0000 7f25ebf87640 1 -- start start 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25ebf87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25ebf87640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25e4109a70 con 0x7f25e407a440 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25e9d23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:08.217 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25e9d23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36458/0 (socket says 172.21.15.49:36458) 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25e9d23640 1 -- 172.21.15.49:0/2103809871 learned_addr learned my addr 172.21.15.49:0/2103809871 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.793+0000 7f25e9d23640 1 -- 172.21.15.49:0/2103809871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e4109bf0 con 0x7f25e407a440 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.794+0000 7f25e9d23640 1 --2- 172.21.15.49:0/2103809871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009820 tx=0x7f25e002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f26b539cfef7f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.794+0000 7f25e8d21640 1 -- 172.21.15.49:0/2103809871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e0037070 con 0x7f25e407a440 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.794+0000 7f25e8d21640 1 -- 172.21.15.49:0/2103809871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f25e00328a0 con 0x7f25e407a440 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.794+0000 7f25e8d21640 1 -- 172.21.15.49:0/2103809871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e0030710 con 0x7f25e407a440 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 -- 172.21.15.49:0/2103809871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 msgr2=0x7f25e4109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 --2- 172.21.15.49:0/2103809871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009820 tx=0x7f25e002a300 comp rx=0 tx=0).stop 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 -- 172.21.15.49:0/2103809871 shutdown_connections 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 --2- 172.21.15.49:0/2103809871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e4109530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 -- 172.21.15.49:0/2103809871 >> 172.21.15.49:0/2103809871 conn(0x7f25e4101160 msgr2=0x7f25e4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 -- 172.21.15.49:0/2103809871 shutdown_connections 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.795+0000 7f25ebf87640 1 -- 172.21.15.49:0/2103809871 wait complete. 2024-09-17T11:37:08.218 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25ebf87640 1 Processor -- start 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25ebf87640 1 -- start start 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25ebf87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25ebf87640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f25e410acb0 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25e9d23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25e9d23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36464/0 (socket says 172.21.15.49:36464) 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25e9d23640 1 -- 172.21.15.49:0/3941860395 learned_addr learned my addr 172.21.15.49:0/3941860395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.796+0000 7f25e9d23640 1 -- 172.21.15.49:0/3941860395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e419ae70 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25e9d23640 1 --2- 172.21.15.49:0/3941860395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25e00097f0 tx=0x7f25e0030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e0037040 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f25e0040070 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e003bd50 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e419b100 con 0x7f25e407a440 2024-09-17T11:37:08.219 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.797+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e419bac0 con 0x7f25e407a440 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 51730+0+0 (secure 0 0 0) 0x7f25e0045460 con 0x7f25e407a440 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25daffd640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 --> v2:172.21.15.49:6800/1073451895 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f25d00422b0 con 0x7f25d003f720 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25e9522640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 msgr2=0x7f25d0041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25e0073c70 con 0x7f25e407a440 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:03.798+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:04.000+0000 7f25e9522640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 msgr2=0x7f25d0041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:04.000+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:04.401+0000 7f25e9522640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 msgr2=0x7f25d0041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:04.401+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:05.202+0000 7f25e9522640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 msgr2=0x7f25d0041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1073451895 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:05.202+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:06.724+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) ==== 51532+0+0 (secure 0 0 0) 0x7f25e0045750 con 0x7f25e407a440 2024-09-17T11:37:08.220 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:06.724+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 msgr2=0x7f25d0041be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:06.724+0000 7f25daffd640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.725+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) ==== 51624+0+0 (secure 0 0 0) 0x7f25e0045ad0 con 0x7f25e407a440 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.725+0000 7f25daffd640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 0x7f25d00453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.725+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 --> v2:172.21.15.49:6800/1315254490 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f25d00422b0 con 0x7f25d0042ff0 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.726+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 0x7f25d00453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.726+0000 7f25e9522640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 0x7f25d00453e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25d4003980 tx=0x7f25d40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:08.221 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.727+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f25d00422b0 con 0x7f25d0042ff0 2024-09-17T11:37:08.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 --> v2:172.21.15.49:6800/1315254490 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f25e41075a0 con 0x7f25d0042ff0 2024-09-17T11:37:08.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25daffd640 1 -- 172.21.15.49:0/3941860395 <== mgr.14118 v2:172.21.15.49:6800/1315254490 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f25e41075a0 con 0x7f25d0042ff0 2024-09-17T11:37:08.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 msgr2=0x7f25d00453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 0x7f25d00453e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25d4003980 tx=0x7f25d40092b0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 msgr2=0x7f25e419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 --2- 172.21.15.49:0/3941860395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25e00097f0 tx=0x7f25e0030800 comp rx=0 tx=0).stop 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 shutdown_connections 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1315254490 conn(0x7f25d0042ff0 0x7f25d00453e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 --2- 172.21.15.49:0/3941860395 >> v2:172.21.15.49:6800/1073451895 conn(0x7f25d003f720 0x7f25d0041be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 --2- 172.21.15.49:0/3941860395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25e407a440 0x7f25e419a910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:08.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 >> 172.21.15.49:0/3941860395 conn(0x7f25e4101160 msgr2=0x7f25e4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:08.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 shutdown_connections 2024-09-17T11:37:08.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:07.734+0000 7f25ebf87640 1 -- 172.21.15.49:0/3941860395 wait complete. 2024-09-17T11:37:08.226 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-09-17T11:37:08.226 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying orchestrator module is enabled... 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: mgrmap e7: y(active, since 1.00528s) 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: Found migration_current of "None". Setting to last migration. 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:08.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:08.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:37:08.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:08 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.639+0000 7f07fa892640 1 Processor -- start 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.639+0000 7f07fa892640 1 -- start start 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.639+0000 7f07fa892640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.639+0000 7f07fa892640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07f407a6b0 con 0x7f07f407a2b0 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.640+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:09.412 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.640+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36530/0 (socket says 172.21.15.49:36530) 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.640+0000 7f07f3fff640 1 -- 172.21.15.49:0/2397933130 learned_addr learned my addr 172.21.15.49:0/2397933130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.640+0000 7f07f3fff640 1 -- 172.21.15.49:0/2397933130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f410bee0 con 0x7f07f407a2b0 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07f3fff640 1 --2- 172.21.15.49:0/2397933130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009820 tx=0x7f07e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a40e24deac204b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07f2ffd640 1 -- 172.21.15.49:0/2397933130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e4037070 con 0x7f07f407a2b0 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07f2ffd640 1 -- 172.21.15.49:0/2397933130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f07e40328a0 con 0x7f07f407a2b0 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07f2ffd640 1 -- 172.21.15.49:0/2397933130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e4030690 con 0x7f07f407a2b0 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07fa892640 1 -- 172.21.15.49:0/2397933130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 msgr2=0x7f07f410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07fa892640 1 --2- 172.21.15.49:0/2397933130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009820 tx=0x7f07e402a270 comp rx=0 tx=0).stop 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07fa892640 1 -- 172.21.15.49:0/2397933130 shutdown_connections 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.641+0000 7f07fa892640 1 --2- 172.21.15.49:0/2397933130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f410b9a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.642+0000 7f07fa892640 1 -- 172.21.15.49:0/2397933130 >> 172.21.15.49:0/2397933130 conn(0x7f07f41011b0 msgr2=0x7f07f41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.642+0000 7f07fa892640 1 -- 172.21.15.49:0/2397933130 shutdown_connections 2024-09-17T11:37:09.413 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.642+0000 7f07fa892640 1 -- 172.21.15.49:0/2397933130 wait complete. 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.642+0000 7f07fa892640 1 Processor -- start 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.642+0000 7f07fa892640 1 -- start start 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07fa892640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07fa892640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410ce70 con 0x7f07f407a2b0 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36534/0 (socket says 172.21.15.49:36534) 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f3fff640 1 -- 172.21.15.49:0/2686398797 learned_addr learned my addr 172.21.15.49:0/2686398797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f3fff640 1 -- 172.21.15.49:0/2686398797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f419cf90 con 0x7f07f407a2b0 2024-09-17T11:37:09.414 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f3fff640 1 --2- 172.21.15.49:0/2686398797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07e4000c00 tx=0x7f07e4032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e4037040 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f07e4040070 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e403be40 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.643+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f419d220 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.644+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f419dc60 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.645+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f410ba20 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.645+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) ==== 51624+0+0 (secure 0 0 0) 0x7f07e403b680 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.645+0000 7f07f17fa640 1 --2- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 0x7f07dc041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.645+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f07e40739c0 con 0x7f07f407a2b0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.645+0000 7f07f37fe640 1 --2- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 0x7f07dc041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.646+0000 7f07f37fe640 1 --2- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 0x7f07dc041ad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:09.415 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.650+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f07e403b430 con 0x7f07f407a2b0 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.884+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "orchestrator"} v 0) -- 0x7f07f419e3d0 con 0x7f07f407a2b0 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.919+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "orchestrator"}]=0 module 'orchestrator' is already enabled (always-on) v8) ==== 143+0+0 (secure 0 0 0) 0x7f07e4037d20 con 0x7f07f407a2b0 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.921+0000 7f07f17fa640 1 -- 172.21.15.49:0/2686398797 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f07e403ad60 con 0x7f07f407a2b0 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 msgr2=0x7f07dc041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 --2- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 0x7f07dc041ad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 msgr2=0x7f07f419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 --2- 172.21.15.49:0/2686398797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07e4000c00 tx=0x7f07e4032740 comp rx=0 tx=0).stop 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 shutdown_connections 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 --2- 172.21.15.49:0/2686398797 >> v2:172.21.15.49:6800/1315254490 conn(0x7f07dc03f610 0x7f07dc041ad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 --2- 172.21.15.49:0/2686398797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f407a2b0 0x7f07f419ca30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 >> 172.21.15.49:0/2686398797 conn(0x7f07f41011b0 msgr2=0x7f07f4101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:09.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 shutdown_connections 2024-09-17T11:37:09.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:08.922+0000 7f07fa892640 1 -- 172.21.15.49:0/2686398797 wait complete. 2024-09-17T11:37:09.417 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-09-17T11:37:10.048 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2686398797' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-17T11:37:10.048 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2686398797' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-17T11:37:10.048 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:09 smithi049 ceph-mon[30017]: mgrmap e8: y(active, since 2s) 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:37:10.045+0000 7f7ee1027640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:11:37:10] ENGINE Error in HTTPServer.serve 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Traceback (most recent call last): 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._connections.run(self.expiration_interval) 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._run(expiration_interval) 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:37:10.353 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s = self.context.wrap_socket( 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: return self.sslsocket_class._create( 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self.do_handshake() 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._sslobj.do_handshake() 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:37:10.354 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:37:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.834+0000 7f174d066640 1 Processor -- start 2024-09-17T11:37:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.834+0000 7f174d066640 1 -- start start 2024-09-17T11:37:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.834+0000 7f174d066640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.834+0000 7f174d066640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1748106210 con 0x7f1748105820 2024-09-17T11:37:10.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.834+0000 7f1746d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.835+0000 7f1746d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36546/0 (socket says 172.21.15.49:36546) 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.835+0000 7f1746d9d640 1 -- 172.21.15.49:0/3773149928 learned_addr learned my addr 172.21.15.49:0/3773149928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.835+0000 7f1746d9d640 1 -- 172.21.15.49:0/3773149928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1748106390 con 0x7f1748105820 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f1746d9d640 1 --2- 172.21.15.49:0/3773149928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1738009820 tx=0x7f173802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d0e8dfe9a3e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f1745d9b640 1 -- 172.21.15.49:0/3773149928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1738037070 con 0x7f1748105820 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f1745d9b640 1 -- 172.21.15.49:0/3773149928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f17380328a0 con 0x7f1748105820 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f1745d9b640 1 -- 172.21.15.49:0/3773149928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1738030690 con 0x7f1748105820 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 -- 172.21.15.49:0/3773149928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 msgr2=0x7f1748105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 --2- 172.21.15.49:0/3773149928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1738009820 tx=0x7f173802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 -- 172.21.15.49:0/3773149928 shutdown_connections 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 --2- 172.21.15.49:0/3773149928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f1748105c20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 -- 172.21.15.49:0/3773149928 >> 172.21.15.49:0/3773149928 conn(0x7f17481010f0 msgr2=0x7f1748103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 -- 172.21.15.49:0/3773149928 shutdown_connections 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.836+0000 7f174d066640 1 -- 172.21.15.49:0/3773149928 wait complete. 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.837+0000 7f174d066640 1 Processor -- start 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.837+0000 7f174d066640 1 -- start start 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.837+0000 7f174d066640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.837+0000 7f174d066640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1748109ca0 con 0x7f1748105820 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f1746d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f1746d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36556/0 (socket says 172.21.15.49:36556) 2024-09-17T11:37:10.620 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f1746d9d640 1 -- 172.21.15.49:0/1054364319 learned_addr learned my addr 172.21.15.49:0/1054364319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f1746d9d640 1 -- 172.21.15.49:0/1054364319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174819aed0 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f1746d9d640 1 --2- 172.21.15.49:0/1054364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1738006f10 tx=0x7f1738032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1738037040 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f1738040070 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174819b160 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f173803a390 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.838+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174819bb20 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.839+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f173803a650 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.839+0000 7f172ffff640 1 --2- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 0x7f1730041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.839+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f17380740b0 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.840+0000 7f174659c640 1 --2- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 0x7f1730041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.840+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748105ca0 con 0x7f1748105820 2024-09-17T11:37:10.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.840+0000 7f174659c640 1 --2- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 0x7f1730041ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f173c0098c0 tx=0x7f173c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:09.845+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f17380313a0 con 0x7f1748105820 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.046+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f174807a790 con 0x7f173003f610 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.051+0000 7f172ffff640 1 -- 172.21.15.49:0/1054364319 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f174807a790 con 0x7f173003f610 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 msgr2=0x7f1730041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 --2- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 0x7f1730041ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f173c0098c0 tx=0x7f173c006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 msgr2=0x7f174819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 --2- 172.21.15.49:0/1054364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1738006f10 tx=0x7f1738032670 comp rx=0 tx=0).stop 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 shutdown_connections 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 --2- 172.21.15.49:0/1054364319 >> v2:172.21.15.49:6800/1315254490 conn(0x7f173003f610 0x7f1730041ad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 --2- 172.21.15.49:0/1054364319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1748105820 0x7f174819a970 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 >> 172.21.15.49:0/1054364319 conn(0x7f17481010f0 msgr2=0x7f174807a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 shutdown_connections 2024-09-17T11:37:10.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:10.054+0000 7f174d066640 1 -- 172.21.15.49:0/1054364319 wait complete. 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:09] ENGINE Bus STARTING 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:09] ENGINE Serving on http://172.21.15.49:8765 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:10] ENGINE Serving on https://172.21.15.49:7150 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:10] ENGINE Bus STARTED 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:10] ENGINE Error in HTTPServer.serve 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: Traceback (most recent call last): 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._connections.run(self.expiration_interval) 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._run(expiration_interval) 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: s = self.context.wrap_socket( 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:37:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout: return self.sslsocket_class._create( 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout: self.do_handshake() 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._sslobj.do_handshake() 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:10.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:10 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:11.809 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:11 smithi049 ceph-mon[30017]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.027+0000 7fb8f6969640 1 Processor -- start 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.027+0000 7fb8f6969640 1 -- start start 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.027+0000 7fb8f6969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.027+0000 7fb8f6969640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0107360 con 0x7fb8f0106970 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.028+0000 7fb8effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.028+0000 7fb8effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36566/0 (socket says 172.21.15.49:36566) 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.028+0000 7fb8effff640 1 -- 172.21.15.49:0/4212392218 learned_addr learned my addr 172.21.15.49:0/4212392218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.028+0000 7fb8effff640 1 -- 172.21.15.49:0/4212392218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f01074e0 con 0x7fb8f0106970 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.028+0000 7fb8effff640 1 --2- 172.21.15.49:0/4212392218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009820 tx=0x7fb8e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8054570040e8752d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:11.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8eeffd640 1 -- 172.21.15.49:0/4212392218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0037070 con 0x7fb8f0106970 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8eeffd640 1 -- 172.21.15.49:0/4212392218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fb8e00328a0 con 0x7fb8f0106970 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8eeffd640 1 -- 172.21.15.49:0/4212392218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0030690 con 0x7fb8f0106970 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 -- 172.21.15.49:0/4212392218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 msgr2=0x7fb8f0106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 --2- 172.21.15.49:0/4212392218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009820 tx=0x7fb8e002a270 comp rx=0 tx=0).stop 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 -- 172.21.15.49:0/4212392218 shutdown_connections 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 --2- 172.21.15.49:0/4212392218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f0106d70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 -- 172.21.15.49:0/4212392218 >> 172.21.15.49:0/4212392218 conn(0x7fb8f0102240 msgr2=0x7fb8f0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 -- 172.21.15.49:0/4212392218 shutdown_connections 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.029+0000 7fb8f6969640 1 -- 172.21.15.49:0/4212392218 wait complete. 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8f6969640 1 Processor -- start 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8f6969640 1 -- start start 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8f6969640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:11.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8f6969640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010adf0 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36572/0 (socket says 172.21.15.49:36572) 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.030+0000 7fb8effff640 1 -- 172.21.15.49:0/3923290882 learned_addr learned my addr 172.21.15.49:0/3923290882 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8effff640 1 -- 172.21.15.49:0/3923290882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f019ae20 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8effff640 1 --2- 172.21.15.49:0/3923290882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e0037040 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fb8e0040070 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f019b0b0 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e003be40 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.031+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f019ba70 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.032+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7fb8e0045460 con 0x7fb8f0106970 2024-09-17T11:37:11.819 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.032+0000 7fb8ed7fa640 1 --2- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 0x7fb8d8041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.032+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f0106df0 con 0x7fb8f0106970 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.032+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7fb8e0073c60 con 0x7fb8f0106970 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.032+0000 7fb8ef7fe640 1 --2- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 0x7fb8d8041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.033+0000 7fb8ef7fe640 1 --2- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 0x7fb8d8041be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098c0 tx=0x7fb8e4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.037+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb8e003b430 con 0x7fb8f0106970 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.225+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb8f0103410 con 0x7fb8d803f720 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.226+0000 7fb8ed7fa640 1 -- 172.21.15.49:0/3923290882 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb8f0103410 con 0x7fb8d803f720 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 msgr2=0x7fb8d8041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 --2- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 0x7fb8d8041be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098c0 tx=0x7fb8e4006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 msgr2=0x7fb8f019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:11.820 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 --2- 172.21.15.49:0/3923290882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0032740 comp rx=0 tx=0).stop 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 shutdown_connections 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 --2- 172.21.15.49:0/3923290882 >> v2:172.21.15.49:6800/1315254490 conn(0x7fb8d803f720 0x7fb8d8041be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 --2- 172.21.15.49:0/3923290882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8f0106970 0x7fb8f019a8c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 >> 172.21.15.49:0/3923290882 conn(0x7fb8f0102240 msgr2=0x7fb8f0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 shutdown_connections 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:11.228+0000 7fb8f6969640 1 -- 172.21.15.49:0/3923290882 wait complete. 2024-09-17T11:37:11.821 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-09-17T11:37:13.024 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-mon[30017]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:13.024 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-mon[30017]: Generating ssh key... 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Generating public/private rsa key pair. 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Your identification has been saved in /tmp/tmppnane48q/key 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Your public key has been saved in /tmp/tmppnane48q/key.pub 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: The key fingerprint is: 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: SHA256:pGKdBbYwjv0ORm20x4oxuuY3ES81fnUgu64W3C7y+yc ceph-f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: The key's randomart image is: 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: +---[RSA 3072]----+ 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | o + | 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | + * =. . | 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | . * = =o . | 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | o.BoB. . . | 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | . ***oSo . | 2024-09-17T11:37:13.025 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | +o+= + | 2024-09-17T11:37:13.026 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | o o.= | 2024-09-17T11:37:13.028 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | o + o E . | 2024-09-17T11:37:13.028 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: | .. =+=.o | 2024-09-17T11:37:13.028 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: +----[SHA256]-----+ 2024-09-17T11:37:13.273 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e5e1b9640 1 Processor -- start 2024-09-17T11:37:13.273 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e5e1b9640 1 -- start start 2024-09-17T11:37:13.273 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e5e1b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:13.273 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e5e1b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58106260 con 0x7f4e58105870 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36580/0 (socket says 172.21.15.49:36580) 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.257+0000 7f4e577fe640 1 -- 172.21.15.49:0/2013326078 learned_addr learned my addr 172.21.15.49:0/2013326078 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.258+0000 7f4e577fe640 1 -- 172.21.15.49:0/2013326078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581063e0 con 0x7f4e58105870 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.258+0000 7f4e577fe640 1 --2- 172.21.15.49:0/2013326078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009820 tx=0x7f4e4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec80d5124a7b102 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e567fc640 1 -- 172.21.15.49:0/2013326078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e48037070 con 0x7f4e58105870 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e567fc640 1 -- 172.21.15.49:0/2013326078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f4e480328a0 con 0x7f4e58105870 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e567fc640 1 -- 172.21.15.49:0/2013326078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e48030690 con 0x7f4e58105870 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/2013326078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 msgr2=0x7f4e58105c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/2013326078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009820 tx=0x7f4e4802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/2013326078 shutdown_connections 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/2013326078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e58105c70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:13.274 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/2013326078 >> 172.21.15.49:0/2013326078 conn(0x7f4e58101140 msgr2=0x7f4e58103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/2013326078 shutdown_connections 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.259+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/2013326078 wait complete. 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e5e1b9640 1 Processor -- start 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e5e1b9640 1 -- start start 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e5e1b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e5e1b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58109cf0 con 0x7f4e58105870 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36592/0 (socket says 172.21.15.49:36592) 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.260+0000 7f4e577fe640 1 -- 172.21.15.49:0/836088632 learned_addr learned my addr 172.21.15.49:0/836088632 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e577fe640 1 -- 172.21.15.49:0/836088632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819ae10 con 0x7f4e58105870 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e577fe640 1 --2- 172.21.15.49:0/836088632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e48037040 con 0x7f4e58105870 2024-09-17T11:37:13.275 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f4e48040070 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4803be40 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819b0a0 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.261+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5819ba60 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.262+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f4e4803b680 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.262+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58105cf0 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.262+0000 7f4e54ff9640 1 --2- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 0x7f4e40041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.262+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f4e48073c20 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.262+0000 7f4e56ffd640 1 --2- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 0x7f4e40041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.263+0000 7f4e56ffd640 1 --2- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 0x7f4e40041be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0098c0 tx=0x7f4e4c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.267+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f4e48036170 con 0x7f4e58105870 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.454+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4e5807a750 con 0x7f4e4003f720 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.732+0000 7f4e54ff9640 1 -- 172.21.15.49:0/836088632 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e5807a750 con 0x7f4e4003f720 2024-09-17T11:37:13.276 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 msgr2=0x7f4e40041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 0x7f4e40041be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0098c0 tx=0x7f4e4c006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 msgr2=0x7f4e5819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/836088632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48032740 comp rx=0 tx=0).stop 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 shutdown_connections 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/836088632 >> v2:172.21.15.49:6800/1315254490 conn(0x7f4e4003f720 0x7f4e40041be0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 --2- 172.21.15.49:0/836088632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e58105870 0x7f4e5819a8b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.734+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 >> 172.21.15.49:0/836088632 conn(0x7f4e58101140 msgr2=0x7f4e5807a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.735+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 shutdown_connections 2024-09-17T11:37:13.277 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:12.735+0000 7f4e5e1b9640 1 -- 172.21.15.49:0/836088632 wait complete. 2024-09-17T11:37:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:13 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:13 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:14.346 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvqi/fE4zhweHoTePDu9d4I6OKhDKDeNmkh+RfUMkrMjRumh2hgh3fLIvJN125Fimtc6lw9i/6X2FSK8FjTedG/gGbxA91t0/TP2zu+2hN7g86rWFNqPdYnNkRtQ8IkvlHz0AFjB3Hqy92UuU+vFmXXtxnf3UcfCG/z6NhW1wICmg5fPmtbjBTv0DalSuixN0SlNLDhOFsfASle7jUwBAqFCd+eJBl7Kf6q+XYSkwMn3tqNyWEYY7tMApZOp69tb+y5rKndHJGMlnchUH8/ZwanNE0kgguxTmh18UXRFWz3AydMyLD7Gb6ujHDcVzYuCzzLCETTzFGY0x46EkBgdY+odTzTxG6LD6KtxUL04C7B/Nhr8P88ICpzESaYcdSXK5+XndIaogDNz0OXo/aOkEdMsxAXTPPYAA00ZLiLaEQfY9U5UyElRV5I46tRv3/h12LZZZ8vn2fKMPUOkt/qKhbInCV1wNcpTu9hd521rK8Pl0idcibn6B4hYLC6L36hcs= ceph-f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.648+0000 7f6efdff6640 1 Processor -- start 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.648+0000 7f6efdff6640 1 -- start start 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6efdff6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6efdff6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8109ad0 con 0x7f6ef807ac00 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44202/0 (socket says 172.21.15.49:44202) 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6ef77fe640 1 -- 172.21.15.49:0/1612486637 learned_addr learned my addr 172.21.15.49:0/1612486637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.649+0000 7f6ef77fe640 1 -- 172.21.15.49:0/1612486637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8109c50 con 0x7f6ef807ac00 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6ef77fe640 1 --2- 172.21.15.49:0/1612486637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009820 tx=0x7f6ee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44e5202d2ba49c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:14.347 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6ef67fc640 1 -- 172.21.15.49:0/1612486637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8037070 con 0x7f6ef807ac00 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6ef67fc640 1 -- 172.21.15.49:0/1612486637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f6ee80328a0 con 0x7f6ef807ac00 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6ef67fc640 1 -- 172.21.15.49:0/1612486637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8030690 con 0x7f6ef807ac00 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6efdff6640 1 -- 172.21.15.49:0/1612486637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 msgr2=0x7f6ef8109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.650+0000 7f6efdff6640 1 --2- 172.21.15.49:0/1612486637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009820 tx=0x7f6ee802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- 172.21.15.49:0/1612486637 shutdown_connections 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 --2- 172.21.15.49:0/1612486637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8109590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- 172.21.15.49:0/1612486637 >> 172.21.15.49:0/1612486637 conn(0x7f6ef8101140 msgr2=0x7f6ef8103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- 172.21.15.49:0/1612486637 shutdown_connections 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- 172.21.15.49:0/1612486637 wait complete. 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 Processor -- start 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- start start 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.651+0000 7f6efdff6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810ac40 con 0x7f6ef807ac00 2024-09-17T11:37:14.348 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44218/0 (socket says 172.21.15.49:44218) 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef77fe640 1 -- 172.21.15.49:0/707633166 learned_addr learned my addr 172.21.15.49:0/707633166 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef77fe640 1 -- 172.21.15.49:0/707633166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8107140 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef77fe640 1 --2- 172.21.15.49:0/707633166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006f10 tx=0x7f6ee8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8037040 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f6ee8040070 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef8105530 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee803a390 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.652+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef8105b10 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.653+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f6ee803b430 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.653+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8107600 con 0x7f6ef807ac00 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.654+0000 7f6ef4ff9640 1 --2- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 0x7f6ee0041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:14.349 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.654+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f6ee8073d40 con 0x7f6ef807ac00 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.654+0000 7f6ef6ffd640 1 --2- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 0x7f6ee0041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.654+0000 7f6ef6ffd640 1 --2- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 0x7f6ee0041be0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0098c0 tx=0x7f6eec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.658+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6ee807a080 con 0x7f6ef807ac00 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.844+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6ef8106160 con 0x7f6ee003f720 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.845+0000 7f6ef4ff9640 1 -- 172.21.15.49:0/707633166 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6ef8106160 con 0x7f6ee003f720 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 msgr2=0x7f6ee0041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 --2- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 0x7f6ee0041be0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0098c0 tx=0x7f6eec006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 msgr2=0x7f6ef8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 --2- 172.21.15.49:0/707633166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006f10 tx=0x7f6ee8032670 comp rx=0 tx=0).stop 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 shutdown_connections 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 --2- 172.21.15.49:0/707633166 >> v2:172.21.15.49:6800/1315254490 conn(0x7f6ee003f720 0x7f6ee0041be0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 --2- 172.21.15.49:0/707633166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ef807ac00 0x7f6ef8106be0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 >> 172.21.15.49:0/707633166 conn(0x7f6ef8101140 msgr2=0x7f6ef8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 shutdown_connections 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:13.847+0000 7f6efdff6640 1 -- 172.21.15.49:0/707633166 wait complete. 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T11:37:14.351 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-09-17T11:37:14.990 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:14 smithi049 ceph-mon[30017]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:15.983 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:15 smithi049 ceph-mon[30017]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:16.830 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:16 smithi049 ceph-mon[30017]: Deploying cephadm binary to smithi049 2024-09-17T11:37:19.362 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.791+0000 7fccc7ff9640 1 Processor -- start 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.791+0000 7fccc7ff9640 1 -- start start 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.791+0000 7fccc7ff9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.791+0000 7fccc7ff9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fccc0107350 con 0x7fccc0106960 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.791+0000 7fccc5d95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.792+0000 7fccc5d95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44220/0 (socket says 172.21.15.49:44220) 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.792+0000 7fccc5d95640 1 -- 172.21.15.49:0/1314588589 learned_addr learned my addr 172.21.15.49:0/1314588589 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.792+0000 7fccc5d95640 1 -- 172.21.15.49:0/1314588589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc01074d0 con 0x7fccc0106960 2024-09-17T11:37:19.363 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.792+0000 7fccc5d95640 1 --2- 172.21.15.49:0/1314588589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009820 tx=0x7fccbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee6262ed413d72e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc4d93640 1 -- 172.21.15.49:0/1314588589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccbc037070 con 0x7fccc0106960 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc4d93640 1 -- 172.21.15.49:0/1314588589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fccbc0328a0 con 0x7fccc0106960 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc4d93640 1 -- 172.21.15.49:0/1314588589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccbc030690 con 0x7fccc0106960 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 -- 172.21.15.49:0/1314588589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 msgr2=0x7fccc0106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/1314588589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009820 tx=0x7fccbc02a270 comp rx=0 tx=0).stop 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 -- 172.21.15.49:0/1314588589 shutdown_connections 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/1314588589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc0106d60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 -- 172.21.15.49:0/1314588589 >> 172.21.15.49:0/1314588589 conn(0x7fccc0102250 msgr2=0x7fccc0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 -- 172.21.15.49:0/1314588589 shutdown_connections 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.793+0000 7fccc7ff9640 1 -- 172.21.15.49:0/1314588589 wait complete. 2024-09-17T11:37:19.364 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc7ff9640 1 Processor -- start 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc7ff9640 1 -- start start 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc7ff9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc7ff9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010ade0 con 0x7fccc0106960 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc5d95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc5d95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44228/0 (socket says 172.21.15.49:44228) 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc5d95640 1 -- 172.21.15.49:0/3647475478 learned_addr learned my addr 172.21.15.49:0/3647475478 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.794+0000 7fccc5d95640 1 -- 172.21.15.49:0/3647475478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc019ae40 con 0x7fccc0106960 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccc5d95640 1 --2- 172.21.15.49:0/3647475478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006f10 tx=0x7fccbc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:19.365 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccbc037040 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7fccbc040070 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc019b0d0 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccbc03a390 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.795+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc019ba90 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7fccbc03b430 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc0106de0 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccb6ffd640 1 --2- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 0x7fccac041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7fccbc073cf0 con 0x7fccc0106960 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccc5594640 1 --2- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 0x7fccac041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:19.366 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.796+0000 7fccc5594640 1 --2- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 0x7fccac041be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccb00098c0 tx=0x7fccb0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.801+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fccbc043360 con 0x7fccc0106960 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:14.989+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) -- 0x7fccc01033b0 con 0x7fccac03f720 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.765+0000 7fccb6ffd640 1 -- 172.21.15.49:0/3647475478 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fccc01033b0 con 0x7fccac03f720 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 msgr2=0x7fccac041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 0x7fccac041be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccb00098c0 tx=0x7fccb0006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 msgr2=0x7fccc019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/3647475478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006f10 tx=0x7fccbc032670 comp rx=0 tx=0).stop 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 shutdown_connections 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/3647475478 >> v2:172.21.15.49:6800/1315254490 conn(0x7fccac03f720 0x7fccac041be0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 --2- 172.21.15.49:0/3647475478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccc0106960 0x7fccc019a8e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 >> 172.21.15.49:0/3647475478 conn(0x7fccc0102250 msgr2=0x7fccc0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:19.367 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 shutdown_connections 2024-09-17T11:37:19.368 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:18.768+0000 7fccc7ff9640 1 -- 172.21.15.49:0/3647475478 wait complete. 2024-09-17T11:37:19.368 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2024-09-17T11:37:20.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:19 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:20.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:19 smithi049 ceph-mon[30017]: Added host smithi049 2024-09-17T11:37:20.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:19 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:20.747 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.792+0000 7f3c79a91640 1 Processor -- start 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.792+0000 7f3c79a91640 1 -- start start 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c79a91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c79a91640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7407a6b0 con 0x7f3c7407a2b0 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44234/0 (socket says 172.21.15.49:44234) 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c737fe640 1 -- 172.21.15.49:0/2368370089 learned_addr learned my addr 172.21.15.49:0/2368370089 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.793+0000 7f3c737fe640 1 -- 172.21.15.49:0/2368370089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7410bee0 con 0x7f3c7407a2b0 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c737fe640 1 --2- 172.21.15.49:0/2368370089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009820 tx=0x7f3c6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af32df902d1e1086 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c727fc640 1 -- 172.21.15.49:0/2368370089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c64037070 con 0x7f3c7407a2b0 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c727fc640 1 -- 172.21.15.49:0/2368370089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f3c640328a0 con 0x7f3c7407a2b0 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c727fc640 1 -- 172.21.15.49:0/2368370089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c64030690 con 0x7f3c7407a2b0 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c79a91640 1 -- 172.21.15.49:0/2368370089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 msgr2=0x7f3c7410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c79a91640 1 --2- 172.21.15.49:0/2368370089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009820 tx=0x7f3c6402a270 comp rx=0 tx=0).stop 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c79a91640 1 -- 172.21.15.49:0/2368370089 shutdown_connections 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c79a91640 1 --2- 172.21.15.49:0/2368370089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7410b9a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.794+0000 7f3c79a91640 1 -- 172.21.15.49:0/2368370089 >> 172.21.15.49:0/2368370089 conn(0x7f3c741011b0 msgr2=0x7f3c741035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.795+0000 7f3c79a91640 1 -- 172.21.15.49:0/2368370089 shutdown_connections 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.795+0000 7f3c79a91640 1 -- 172.21.15.49:0/2368370089 wait complete. 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.795+0000 7f3c79a91640 1 Processor -- start 2024-09-17T11:37:20.777 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.795+0000 7f3c79a91640 1 -- start start 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c79a91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c79a91640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410ce70 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44246/0 (socket says 172.21.15.49:44246) 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c737fe640 1 -- 172.21.15.49:0/1855997059 learned_addr learned my addr 172.21.15.49:0/1855997059 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c737fe640 1 -- 172.21.15.49:0/1855997059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7419cf90 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c737fe640 1 --2- 172.21.15.49:0/1855997059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006f10 tx=0x7f3c64032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c64037040 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7f3c64040070 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c6403a390 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7419d220 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.796+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7419dc60 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.797+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f3c6403b430 con 0x7f3c7407a2b0 2024-09-17T11:37:20.778 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.797+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7410ba20 con 0x7f3c7407a2b0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.798+0000 7f3c70ff9640 1 --2- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 0x7f3c5c041b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.798+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f3c64073dd0 con 0x7f3c7407a2b0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.798+0000 7f3c72ffd640 1 --2- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 0x7f3c5c041b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.798+0000 7f3c72ffd640 1 --2- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 0x7f3c5c041b90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098c0 tx=0x7f3c68006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:19.802+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3c64041270 con 0x7f3c7407a2b0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.024+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3c7419e500 con 0x7f3c5c03f6d0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.029+0000 7f3c70ff9640 1 -- 172.21.15.49:0/1855997059 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3c7419e500 con 0x7f3c5c03f6d0 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 msgr2=0x7f3c5c041b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 --2- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 0x7f3c5c041b90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098c0 tx=0x7f3c68006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 msgr2=0x7f3c7419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 --2- 172.21.15.49:0/1855997059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006f10 tx=0x7f3c64032670 comp rx=0 tx=0).stop 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 shutdown_connections 2024-09-17T11:37:20.779 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 --2- 172.21.15.49:0/1855997059 >> v2:172.21.15.49:6800/1315254490 conn(0x7f3c5c03f6d0 0x7f3c5c041b90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:20.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 --2- 172.21.15.49:0/1855997059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c7407a2b0 0x7f3c7419ca30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:20.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 >> 172.21.15.49:0/1855997059 conn(0x7f3c741011b0 msgr2=0x7f3c74101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:20.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 shutdown_connections 2024-09-17T11:37:20.780 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:20.031+0000 7f3c79a91640 1 -- 172.21.15.49:0/1855997059 wait complete. 2024-09-17T11:37:20.780 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2024-09-17T11:37:21.283 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:21 smithi049 ceph-mon[30017]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:21.283 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:21 smithi049 ceph-mon[30017]: Saving service mon spec with placement count:5 2024-09-17T11:37:21.283 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:21 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:22.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T11:37:22.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.695+0000 7ff2c1012640 1 Processor -- start 2024-09-17T11:37:22.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.695+0000 7ff2c1012640 1 -- start start 2024-09-17T11:37:22.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.695+0000 7ff2c1012640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.695+0000 7ff2c1012640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc07d190 con 0x7ff2bc07cce0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.696+0000 7ff2bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.696+0000 7ff2bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44250/0 (socket says 172.21.15.49:44250) 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.696+0000 7ff2bad9d640 1 -- 172.21.15.49:0/1818851794 learned_addr learned my addr 172.21.15.49:0/1818851794 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.696+0000 7ff2bad9d640 1 -- 172.21.15.49:0/1818851794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc07b880 con 0x7ff2bc07cce0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2bad9d640 1 --2- 172.21.15.49:0/1818851794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e80 tx=0x7ff2ac02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201aa17d6856495c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2b9d9b640 1 -- 172.21.15.49:0/1818851794 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac036070 con 0x7ff2bc07cce0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2b9d9b640 1 -- 172.21.15.49:0/1818851794 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7ff2ac031860 con 0x7ff2bc07cce0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2b9d9b640 1 -- 172.21.15.49:0/1818851794 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac007880 con 0x7ff2bc07cce0 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 -- 172.21.15.49:0/1818851794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 msgr2=0x7ff2bc07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 --2- 172.21.15.49:0/1818851794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e80 tx=0x7ff2ac02a7a0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 -- 172.21.15.49:0/1818851794 shutdown_connections 2024-09-17T11:37:22.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 --2- 172.21.15.49:0/1818851794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc07b340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 -- 172.21.15.49:0/1818851794 >> 172.21.15.49:0/1818851794 conn(0x7ff2bc102250 msgr2=0x7ff2bc104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 -- 172.21.15.49:0/1818851794 shutdown_connections 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.697+0000 7ff2c1012640 1 -- 172.21.15.49:0/1818851794 wait complete. 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.698+0000 7ff2c1012640 1 Processor -- start 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.698+0000 7ff2c1012640 1 -- start start 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.698+0000 7ff2c1012640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.698+0000 7ff2c1012640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc10ceb0 con 0x7ff2bc07cce0 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44264/0 (socket says 172.21.15.49:44264) 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2bad9d640 1 -- 172.21.15.49:0/2832903661 learned_addr learned my addr 172.21.15.49:0/2832903661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2bad9d640 1 -- 172.21.15.49:0/2832903661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19d000 con 0x7ff2bc07cce0 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2bad9d640 1 --2- 172.21.15.49:0/2832903661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac008dc0 tx=0x7ff2ac008df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac041020 con 0x7ff2bc07cce0 2024-09-17T11:37:22.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc19d290 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1023+0+0 (secure 0 0 0) 0x7ff2ac031a10 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ac036040 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.699+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc19dcd0 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.700+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7ff2ac044020 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.700+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc07b3c0 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.700+0000 7ff2a3fff640 1 --2- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 0x7ff2a4041b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.700+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7ff2ac075330 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.701+0000 7ff2ba59c640 1 --2- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 0x7ff2a4041b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.701+0000 7ff2ba59c640 1 --2- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 0x7ff2a4041b90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098c0 tx=0x7ff2b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.705+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff2ac04b020 con 0x7ff2bc07cce0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.905+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 --> v2:172.21.15.49:6800/1315254490 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff2bc103410 con 0x7ff2a403f6d0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.909+0000 7ff2a3fff640 1 -- 172.21.15.49:0/2832903661 <== mgr.14118 v2:172.21.15.49:6800/1315254490 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff2bc103410 con 0x7ff2a403f6d0 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 msgr2=0x7ff2a4041b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:22.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 --2- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 0x7ff2a4041b90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098c0 tx=0x7ff2b0006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 msgr2=0x7ff2bc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 --2- 172.21.15.49:0/2832903661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac008dc0 tx=0x7ff2ac008df0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 shutdown_connections 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 --2- 172.21.15.49:0/2832903661 >> v2:172.21.15.49:6800/1315254490 conn(0x7ff2a403f6d0 0x7ff2a4041b90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 --2- 172.21.15.49:0/2832903661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2bc07cce0 0x7ff2bc19caa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 >> 172.21.15.49:0/2832903661 conn(0x7ff2bc102250 msgr2=0x7ff2bc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 shutdown_connections 2024-09-17T11:37:22.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:21.911+0000 7ff2c1012640 1 -- 172.21.15.49:0/2832903661 wait complete. 2024-09-17T11:37:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:22 smithi049 ceph-mon[30017]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:22 smithi049 ceph-mon[30017]: Saving service mgr spec with placement count:2 2024-09-17T11:37:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:22 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:22 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:22 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:23.985 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.040+0000 7f335574b640 1 Processor -- start 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.040+0000 7f335574b640 1 -- start start 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.040+0000 7f335574b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.040+0000 7f335574b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3350109ae0 con 0x7f335007a3c0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.041+0000 7f334effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.041+0000 7f334effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44266/0 (socket says 172.21.15.49:44266) 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.041+0000 7f334effd640 1 -- 172.21.15.49:0/1115176035 learned_addr learned my addr 172.21.15.49:0/1115176035 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.041+0000 7f334effd640 1 -- 172.21.15.49:0/1115176035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350109c60 con 0x7f335007a3c0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f334effd640 1 --2- 172.21.15.49:0/1115176035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3348009820 tx=0x7f334802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a656fede99943fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f334dffb640 1 -- 172.21.15.49:0/1115176035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3348037070 con 0x7f335007a3c0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f334dffb640 1 -- 172.21.15.49:0/1115176035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f33480328c0 con 0x7f335007a3c0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f334dffb640 1 -- 172.21.15.49:0/1115176035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3348030640 con 0x7f335007a3c0 2024-09-17T11:37:23.986 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 -- 172.21.15.49:0/1115176035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 msgr2=0x7f33501095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 --2- 172.21.15.49:0/1115176035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3348009820 tx=0x7f334802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 -- 172.21.15.49:0/1115176035 shutdown_connections 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 --2- 172.21.15.49:0/1115176035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f33501095a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 -- 172.21.15.49:0/1115176035 >> 172.21.15.49:0/1115176035 conn(0x7f33501011b0 msgr2=0x7f33501035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 -- 172.21.15.49:0/1115176035 shutdown_connections 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.042+0000 7f335574b640 1 -- 172.21.15.49:0/1115176035 wait complete. 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f335574b640 1 Processor -- start 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f335574b640 1 -- start start 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f335574b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f335574b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f335010ad00 con 0x7f335007a3c0 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f334effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f334effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44274/0 (socket says 172.21.15.49:44274) 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.043+0000 7f334effd640 1 -- 172.21.15.49:0/3118850727 learned_addr learned my addr 172.21.15.49:0/3118850727 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f334effd640 1 -- 172.21.15.49:0/3118850727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335019ad80 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f334effd640 1 --2- 172.21.15.49:0/3118850727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c00 tx=0x7f3348030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3348037040 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3348040070 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335019b010 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f334803a390 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.044+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335019b9d0 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.045+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f334803a5c0 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.045+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3350107590 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.045+0000 7f3347fff640 1 --2- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 0x7f333c041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.045+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f3348073fd0 con 0x7f335007a3c0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.045+0000 7f334e7fc640 1 --2- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 0x7f333c041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:23.988 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.046+0000 7f334e7fc640 1 --2- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 0x7f333c041be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33380098c0 tx=0x7f3338006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.050+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f334803a8b0 con 0x7f335007a3c0 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.252+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f335019c110 con 0x7f335007a3c0 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.255+0000 7f3347fff640 1 -- 172.21.15.49:0/3118850727 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3348073d30 con 0x7f335007a3c0 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 msgr2=0x7f333c041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 --2- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 0x7f333c041be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33380098c0 tx=0x7f3338006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 msgr2=0x7f335019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 --2- 172.21.15.49:0/3118850727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c00 tx=0x7f3348030b70 comp rx=0 tx=0).stop 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 shutdown_connections 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 --2- 172.21.15.49:0/3118850727 >> v2:172.21.15.49:6800/1315254490 conn(0x7f333c03f720 0x7f333c041be0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 --2- 172.21.15.49:0/3118850727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f335007a3c0 0x7f335019a820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 >> 172.21.15.49:0/3118850727 conn(0x7f33501011b0 msgr2=0x7f3350101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 shutdown_connections 2024-09-17T11:37:23.989 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:23.257+0000 7f335574b640 1 -- 172.21.15.49:0/3118850727 wait complete. 2024-09-17T11:37:24.547 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3118850727' entity='client.admin' 2024-09-17T11:37:25.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:25 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:25.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:25 smithi049 ceph-mon[30017]: from='mgr.14118 172.21.15.49:0/2932232191' entity='mgr.y' 2024-09-17T11:37:25.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1489117005' entity='client.admin' 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.761+0000 7f48c692a640 1 Processor -- start 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.761+0000 7f48c692a640 1 -- start start 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.761+0000 7f48c692a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.761+0000 7f48c692a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f48c0106250 con 0x7f48c0105860 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.761+0000 7f48bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.762+0000 7f48bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42880/0 (socket says 172.21.15.49:42880) 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.762+0000 7f48bffff640 1 -- 172.21.15.49:0/495826226 learned_addr learned my addr 172.21.15.49:0/495826226 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.762+0000 7f48bffff640 1 -- 172.21.15.49:0/495826226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c01063d0 con 0x7f48c0105860 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.762+0000 7f48bffff640 1 --2- 172.21.15.49:0/495826226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009820 tx=0x7f48b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19caf439cfa4a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:25.502 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48beffd640 1 -- 172.21.15.49:0/495826226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b0037070 con 0x7f48c0105860 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48beffd640 1 -- 172.21.15.49:0/495826226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f48b00328c0 con 0x7f48c0105860 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48beffd640 1 -- 172.21.15.49:0/495826226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b0030640 con 0x7f48c0105860 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 -- 172.21.15.49:0/495826226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 msgr2=0x7f48c0105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 --2- 172.21.15.49:0/495826226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009820 tx=0x7f48b002a270 comp rx=0 tx=0).stop 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 -- 172.21.15.49:0/495826226 shutdown_connections 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 --2- 172.21.15.49:0/495826226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c0105c60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 -- 172.21.15.49:0/495826226 >> 172.21.15.49:0/495826226 conn(0x7f48c01011b0 msgr2=0x7f48c01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 -- 172.21.15.49:0/495826226 shutdown_connections 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.763+0000 7f48c692a640 1 -- 172.21.15.49:0/495826226 wait complete. 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.764+0000 7f48c692a640 1 Processor -- start 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.764+0000 7f48c692a640 1 -- start start 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.764+0000 7f48c692a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.764+0000 7f48c692a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f48c0109ce0 con 0x7f48c0105860 2024-09-17T11:37:25.503 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42884/0 (socket says 172.21.15.49:42884) 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bffff640 1 -- 172.21.15.49:0/1489117005 learned_addr learned my addr 172.21.15.49:0/1489117005 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bffff640 1 -- 172.21.15.49:0/1489117005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c0190d50 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bffff640 1 --2- 172.21.15.49:0/1489117005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b0037040 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f48b0040070 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c018f140 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b003a390 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.765+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c018f6c0 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.767+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f48b003a5c0 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.767+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c0105c60 con 0x7f48c0105860 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.767+0000 7f48bd7fa640 1 --2- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 0x7f48a8041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:25.504 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.767+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f48b0073ed0 con 0x7f48c0105860 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.767+0000 7f48bf7fe640 1 --2- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 0x7f48a8041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.768+0000 7f48bf7fe640 1 --2- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 0x7f48a8041be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098c0 tx=0x7f48b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.772+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f48b002e200 con 0x7f48c0105860 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.961+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f48c018fc60 con 0x7f48c0105860 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.964+0000 7f48bd7fa640 1 -- 172.21.15.49:0/1489117005 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f48b0043e40 con 0x7f48c0105860 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 msgr2=0x7f48a8041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 --2- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 0x7f48a8041be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098c0 tx=0x7f48b4006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 msgr2=0x7f48c01907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 --2- 172.21.15.49:0/1489117005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0030b70 comp rx=0 tx=0).stop 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 shutdown_connections 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 --2- 172.21.15.49:0/1489117005 >> v2:172.21.15.49:6800/1315254490 conn(0x7f48a803f720 0x7f48a8041be0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 --2- 172.21.15.49:0/1489117005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48c0105860 0x7f48c01907f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 >> 172.21.15.49:0/1489117005 conn(0x7f48c01011b0 msgr2=0x7f48c0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:25.505 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 shutdown_connections 2024-09-17T11:37:25.506 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:24.966+0000 7f48c692a640 1 -- 172.21.15.49:0/1489117005 wait complete. 2024-09-17T11:37:25.506 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-09-17T11:37:26.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/598721264' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-17T11:37:26.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setuser ceph since I am not root 2024-09-17T11:37:26.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setgroup ceph since I am not root 2024-09-17T11:37:27.540 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.128+0000 7f4170aad640 1 Processor -- start 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f4170aad640 1 -- start start 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f4170aad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f4170aad640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f416c106720 con 0x7f416c105de0 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f416a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f416a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42896/0 (socket says 172.21.15.49:42896) 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f416a59c640 1 -- 172.21.15.49:0/327388693 learned_addr learned my addr 172.21.15.49:0/327388693 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.129+0000 7f416a59c640 1 -- 172.21.15.49:0/327388693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1068a0 con 0x7f416c105de0 2024-09-17T11:37:27.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.130+0000 7f416a59c640 1 --2- 172.21.15.49:0/327388693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4164009820 tx=0x7f416402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce5ad7422288989 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.130+0000 7f416959a640 1 -- 172.21.15.49:0/327388693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4164037070 con 0x7f416c105de0 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.130+0000 7f416959a640 1 -- 172.21.15.49:0/327388693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f41640328c0 con 0x7f416c105de0 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.130+0000 7f416959a640 1 -- 172.21.15.49:0/327388693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4164030640 con 0x7f416c105de0 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 -- 172.21.15.49:0/327388693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 msgr2=0x7f416c1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 --2- 172.21.15.49:0/327388693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4164009820 tx=0x7f416402a270 comp rx=0 tx=0).stop 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 -- 172.21.15.49:0/327388693 shutdown_connections 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 --2- 172.21.15.49:0/327388693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1061e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 -- 172.21.15.49:0/327388693 >> 172.21.15.49:0/327388693 conn(0x7f416c1008e0 msgr2=0x7f416c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 -- 172.21.15.49:0/327388693 shutdown_connections 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.131+0000 7f4170aad640 1 -- 172.21.15.49:0/327388693 wait complete. 2024-09-17T11:37:27.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.132+0000 7f4170aad640 1 Processor -- start 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.132+0000 7f4170aad640 1 -- start start 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.132+0000 7f4170aad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.132+0000 7f4170aad640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1079a0 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f416a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f416a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42898/0 (socket says 172.21.15.49:42898) 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f416a59c640 1 -- 172.21.15.49:0/598721264 learned_addr learned my addr 172.21.15.49:0/598721264 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f416a59c640 1 -- 172.21.15.49:0/598721264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c190d00 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f416a59c640 1 --2- 172.21.15.49:0/598721264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416402a830 tx=0x7f4164030bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4164037040 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f4164040070 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c18f0f0 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.133+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f416403be40 con 0x7f416c105de0 2024-09-17T11:37:27.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.134+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c18f670 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.135+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c1061e0 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.135+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 51730+0+0 (secure 0 0 0) 0x7f416403a4a0 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.135+0000 7f41637fe640 1 --2- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 0x7f4158041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.135+0000 7f4169d9b640 1 --2- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 0x7f4158041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.136+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f4164073e80 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.136+0000 7f4169d9b640 1 --2- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 0x7f4158041be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41540098c0 tx=0x7f4154006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.140+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f41640328c0 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.383+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f416c18fba0 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.435+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f4164073be0 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.436+0000 7f41637fe640 1 -- 172.21.15.49:0/598721264 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 9) ==== 51743+0+0 (secure 0 0 0) 0x7f4164045460 con 0x7f416c105de0 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.437+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 msgr2=0x7f4158041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.437+0000 7f4170aad640 1 --2- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 0x7f4158041be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41540098c0 tx=0x7f4154006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:27.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.437+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 msgr2=0x7f416c1907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.437+0000 7f4170aad640 1 --2- 172.21.15.49:0/598721264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416402a830 tx=0x7f4164030bc0 comp rx=0 tx=0).stop 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 shutdown_connections 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 --2- 172.21.15.49:0/598721264 >> v2:172.21.15.49:6800/1315254490 conn(0x7f415803f720 0x7f4158041be0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 --2- 172.21.15.49:0/598721264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f416c105de0 0x7f416c1907a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 >> 172.21.15.49:0/598721264 conn(0x7f416c1008e0 msgr2=0x7f416c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 shutdown_connections 2024-09-17T11:37:27.545 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:26.438+0000 7f4170aad640 1 -- 172.21.15.49:0/598721264 wait complete. 2024-09-17T11:37:27.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/598721264' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T11:37:27.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:27 smithi049 ceph-mon[30017]: mgrmap e9: y(active, since 19s) 2024-09-17T11:37:28.785 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3817807535' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.976+0000 7f577428d640 1 Processor -- start 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.976+0000 7f577428d640 1 -- start start 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f577428d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f577428d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f576c109ae0 con 0x7f576c07a3c0 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f5772029640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:28.791 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f5772029640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42926/0 (socket says 172.21.15.49:42926) 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f5772029640 1 -- 172.21.15.49:0/4130262599 learned_addr learned my addr 172.21.15.49:0/4130262599 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.977+0000 7f5772029640 1 -- 172.21.15.49:0/4130262599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c109c60 con 0x7f576c07a3c0 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.978+0000 7f5772029640 1 --2- 172.21.15.49:0/4130262599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5768009820 tx=0x7f576802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4966afc8ecf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.978+0000 7f5771027640 1 -- 172.21.15.49:0/4130262599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5768037070 con 0x7f576c07a3c0 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.978+0000 7f5771027640 1 -- 172.21.15.49:0/4130262599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f57680328c0 con 0x7f576c07a3c0 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.978+0000 7f5771027640 1 -- 172.21.15.49:0/4130262599 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5768030640 con 0x7f576c07a3c0 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- 172.21.15.49:0/4130262599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 msgr2=0x7f576c1095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 --2- 172.21.15.49:0/4130262599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5768009820 tx=0x7f576802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- 172.21.15.49:0/4130262599 shutdown_connections 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 --2- 172.21.15.49:0/4130262599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c1095a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- 172.21.15.49:0/4130262599 >> 172.21.15.49:0/4130262599 conn(0x7f576c1011b0 msgr2=0x7f576c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- 172.21.15.49:0/4130262599 shutdown_connections 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- 172.21.15.49:0/4130262599 wait complete. 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 Processor -- start 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- start start 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.979+0000 7f577428d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10ad00 con 0x7f576c07a3c0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f5772029640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f5772029640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42942/0 (socket says 172.21.15.49:42942) 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f5772029640 1 -- 172.21.15.49:0/3817807535 learned_addr learned my addr 172.21.15.49:0/3817807535 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f5772029640 1 -- 172.21.15.49:0/3817807535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c19ad80 con 0x7f576c07a3c0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f5772029640 1 --2- 172.21.15.49:0/3817807535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5768000c00 tx=0x7f5768030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5768037040 con 0x7f576c07a3c0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f5768040070 con 0x7f576c07a3c0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c19b010 con 0x7f576c07a3c0 2024-09-17T11:37:28.793 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576803a390 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.980+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c19b9d0 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.981+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 51743+0+0 (secure 0 0 0) 0x7f576803b430 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.981+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c107590 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.981+0000 7f57637fe640 1 --2- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 0x7f5758041c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.981+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7f5768073ed0 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.981+0000 7f5771828640 1 -- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 msgr2=0x7f5758041c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.982+0000 7f5771828640 1 --2- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 0x7f5758041c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:27.986+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f5768073c30 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.183+0000 7f5771828640 1 -- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 msgr2=0x7f5758041c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.183+0000 7f5771828640 1 --2- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 0x7f5758041c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.220+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f576c19c110 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.220+0000 7f57637fe640 1 -- 172.21.15.49:0/3817807535 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f57680327b0 con 0x7f576c07a3c0 2024-09-17T11:37:28.794 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.222+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 msgr2=0x7f5758041c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.222+0000 7f577428d640 1 --2- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 0x7f5758041c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.222+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 msgr2=0x7f576c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.222+0000 7f577428d640 1 --2- 172.21.15.49:0/3817807535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5768000c00 tx=0x7f5768030b70 comp rx=0 tx=0).stop 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 shutdown_connections 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 --2- 172.21.15.49:0/3817807535 >> v2:172.21.15.49:6800/1315254490 conn(0x7f575803f770 0x7f5758041c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 --2- 172.21.15.49:0/3817807535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576c07a3c0 0x7f576c19a820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 >> 172.21.15.49:0/3817807535 conn(0x7f576c1011b0 msgr2=0x7f576c101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 shutdown_connections 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:28.223+0000 7f577428d640 1 -- 172.21.15.49:0/3817807535 wait complete. 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-09-17T11:37:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: Active manager daemon y restarted 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: Activating manager daemon y 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: mgrmap e10: y(active, starting, since 0.00344055s) 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:37:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:37:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:31 smithi049 ceph-mon[30017]: Manager daemon y is now available 2024-09-17T11:37:33.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:32 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:33.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:32 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:37:33.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:32 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:37:33.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:32 smithi049 ceph-mon[30017]: mgrmap e11: y(active, since 1.0069s) 2024-09-17T11:37:33.305 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-09-17T11:37:33.305 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T11:37:33.305 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4b52d9640 1 Processor -- start 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4b52d9640 1 -- start start 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4b52d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4b52d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0109a70 con 0x7fd4b007ac60 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42946/0 (socket says 172.21.15.49:42946) 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4aeffd640 1 -- 172.21.15.49:0/100343099 learned_addr learned my addr 172.21.15.49:0/100343099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.341+0000 7fd4aeffd640 1 -- 172.21.15.49:0/100343099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b0109bf0 con 0x7fd4b007ac60 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.342+0000 7fd4aeffd640 1 --2- 172.21.15.49:0/100343099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009820 tx=0x7fd4a002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228ed9f416d38347 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4adffb640 1 -- 172.21.15.49:0/100343099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0037070 con 0x7fd4b007ac60 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4adffb640 1 -- 172.21.15.49:0/100343099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fd4a00328c0 con 0x7fd4b007ac60 2024-09-17T11:37:33.306 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4adffb640 1 -- 172.21.15.49:0/100343099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a00306d0 con 0x7fd4b007ac60 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4b52d9640 1 -- 172.21.15.49:0/100343099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 msgr2=0x7fd4b0109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/100343099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009820 tx=0x7fd4a002a300 comp rx=0 tx=0).stop 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4b52d9640 1 -- 172.21.15.49:0/100343099 shutdown_connections 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/100343099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b0109530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.343+0000 7fd4b52d9640 1 -- 172.21.15.49:0/100343099 >> 172.21.15.49:0/100343099 conn(0x7fd4b0101160 msgr2=0x7fd4b0103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.344+0000 7fd4b52d9640 1 -- 172.21.15.49:0/100343099 shutdown_connections 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.344+0000 7fd4b52d9640 1 -- 172.21.15.49:0/100343099 wait complete. 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.344+0000 7fd4b52d9640 1 Processor -- start 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.344+0000 7fd4b52d9640 1 -- start start 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4b52d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4b52d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010acf0 con 0x7fd4b007ac60 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:33.307 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42954/0 (socket says 172.21.15.49:42954) 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4aeffd640 1 -- 172.21.15.49:0/297925717 learned_addr learned my addr 172.21.15.49:0/297925717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4aeffd640 1 -- 172.21.15.49:0/297925717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b019d030 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4aeffd640 1 --2- 172.21.15.49:0/297925717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a0030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0037040 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b019d2c0 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fd4a0040070 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a003bda0 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.345+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b019dc80 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 51743+0+0 (secure 0 0 0) 0x7fd4a0045460 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd497fff640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 --> v2:172.21.15.49:6800/1315254490 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd4980422b0 con 0x7fd49803f720 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (secure 0 0 0) 0x7fd4a0073e90 con 0x7fd4b007ac60 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd4ae7fc640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 msgr2=0x7fd498041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:33.308 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.347+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.548+0000 7fd4ae7fc640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 msgr2=0x7fd498041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.548+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.949+0000 7fd4ae7fc640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 msgr2=0x7fd498041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:29.949+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:30.750+0000 7fd4ae7fc640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 msgr2=0x7fd498041be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1315254490 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:30.750+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:31.686+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) ==== 51545+0+0 (secure 0 0 0) 0x7fd4a0072c90 con 0x7fd4b007ac60 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:31.686+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 msgr2=0x7fd498041be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:31.686+0000 7fd497fff640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.688+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) ==== 51637+0+0 (secure 0 0 0) 0x7fd4a0045b20 con 0x7fd4b007ac60 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.689+0000 7fd497fff640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 0x7fd4980453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.689+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 --> v2:172.21.15.49:6800/1112797402 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd4a002ce50 con 0x7fd498042ff0 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.689+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 0x7fd4980453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:33.309 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.690+0000 7fd4ae7fc640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 0x7fd4980453e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4003980 tx=0x7fd4a4006e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.691+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd4a002ce50 con 0x7fd498042ff0 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.702+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 --> v2:172.21.15.49:6800/1112797402 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd4b0107520 con 0x7fd498042ff0 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd497fff640 1 -- 172.21.15.49:0/297925717 <== mgr.14152 v2:172.21.15.49:6800/1112797402 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd4b0107520 con 0x7fd498042ff0 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 msgr2=0x7fd4980453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 0x7fd4980453e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4003980 tx=0x7fd4a4006e10 comp rx=0 tx=0).stop 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 msgr2=0x7fd4b019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/297925717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a0030a60 comp rx=0 tx=0).stop 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 shutdown_connections 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1112797402 conn(0x7fd498042ff0 0x7fd4980453e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/297925717 >> v2:172.21.15.49:6800/1315254490 conn(0x7fd49803f720 0x7fd498041be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 --2- 172.21.15.49:0/297925717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4b007ac60 0x7fd4b019cad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.703+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 >> 172.21.15.49:0/297925717 conn(0x7fd4b0101160 msgr2=0x7fd4b0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:33.310 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.704+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 shutdown_connections 2024-09-17T11:37:33.311 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:32.704+0000 7fd4b52d9640 1 -- 172.21.15.49:0/297925717 wait complete. 2024-09-17T11:37:33.311 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-09-17T11:37:33.311 INFO:teuthology.orchestra.run.smithi049.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-17T11:37:33.797 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-mon[30017]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T11:37:33.798 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-mon[30017]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:37:33.933+0000 7f3f47690640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:11:37:33] ENGINE Error in HTTPServer.serve 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Traceback (most recent call last): 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._connections.run(self.expiration_interval) 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:37:34.085 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._run(expiration_interval) 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s = self.context.wrap_socket( 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: return self.sslsocket_class._create( 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self.do_handshake() 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._sslobj.do_handshake() 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:37:34.086 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:37:33 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:33] ENGINE Bus STARTING 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: mgrmap e12: y(active, since 2s) 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:33] ENGINE Serving on http://172.21.15.49:8765 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:33] ENGINE Serving on https://172.21.15.49:7150 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:33] ENGINE Bus STARTED 2024-09-17T11:37:35.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: [17/Sep/2024:11:37:33] ENGINE Error in HTTPServer.serve 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: Traceback (most recent call last): 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._connections.run(self.expiration_interval) 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._run(expiration_interval) 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: s = self.context.wrap_socket( 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:37:35.033 INFO:journalctl@ceph.mon.a.smithi049.stdout: return self.sslsocket_class._create( 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout: self.do_handshake() 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._sslobj.do_handshake() 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:35.034 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:34 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUR2OtN26x0Docr3hYDUhE8Iz5sC0wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxNzExMzczNFoXDTM0\nMDkxODExMzczNFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQ5MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAxKGDV2PbtbnQuCcSkVyhTLaYnwgyD3vBZOOo\nFOfSdKIOt80Dh/Ifre4/s9yXEGSfWDVco29PXj7dzA9tmd1h35w44Bbkcx6VOUT0\n/ChEseyuVL9rNpz4gg/94GkgK1RWafPn+cgqITWWmNKeYuccZEk11krheCt5M3mp\nz0F7FoD0SeSAAEIE/n1PT6FKDsLqVwAilJWiYBhb9MamnJ0ghFMirNDdSJT8KvRI\n8kfIb+TICmofV81GmbTyQ+1Iq4476QnqTVV7kkL/YrjpJlnxlrwR1GIeaHD+D1dH\nakIdoVBWSumy1NZMI6OhuLEI5HCxuZFQ/xo/CTWryjuKSym5OQ+B3nykpnQJtmsG\nmSQC4XE5llYLBR+xRyiozGykRnrRRlULVGepENfREmJfLzjXPLg7xLrGxsxef5mS\nS7qlaqEC/sYhpipaOKZ4mIHtpp9uZdnsD3dBabx+HKc/PT/4Q1Y0iIoHPLQ2n7zH\nfNjHlNSbgxE3N6FdvZSfpoijRCq9HYA+KjiHQMNKTsdi9e1IDbEo9UMWq1GHd31M\nou7dSGGyRTMHTz3KMyJiXo4MtgNVidJUldOqWekufNJEZc7ozMCTz8WLHAOtaHzd\nBPwi7AmkE50kOOqVzRD7kXEW1ABGSucBA4VextBPX7G1XaCtU7bkW2qXbaXVXVVK\nWOx7sR8CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQ5ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPMTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAs\nuZ2kPaM+lOSF2DpSWtKUaxDWkjFWSfDysHkD4M4QClUig0TfJQrRjdDuDJA226PT\n+wpPRBAdC6WaKBiSZf1lV/W/quuJp0e+/468xb6VDKwgjSsliwuWW3MLI/cwwnXs\n5LaadwNTiaEa3NEtuAJGZ+wsAkj3K02lXeepyN2GR2doIFCKeVOK/oPoJRyjOCVc\nAx9u98lm4My2y6ZGttbVgTQM6FOscrv8StluvfKV8wNbvyjM8rXifKzeF+WIkzJi\nD7nv9lZN7DIAEkLLk8A69wNchUEgBmPuijx2bnPSIGIye1dYCG3+mPK5I2md1kw8\nFiG8w1b9Kqi5+iOkzYyFgvmLwNMSz5QaewxBsaOQSDQJpA8UxCE5+JhYrBqpE/1w\nwo1FzSAEkE4iHMlO23fNovGyRKtiDoX2FIw2WnPb0mpP69tKWFWaVu6EZjf3GhmK\nhpaYzSAsboA3qjnt4TqyOZTNA8RSfgp8AHOMTJOHrn9l7IYfWEVsqmQHbH6swnZs\nTVeDL5TF6B76U6V1wpfBWbUDj6RiYck5nzPHICjG4NiijHXUSIdkN09Ao4XwMt30\n26qwN8OSKczy5EbkfrS9iFhHpJW//Y7P5B/48HEtbNoorwc++gKp2hA31HAZVHdY\nbHerMqAkvaQ30AZTXnhNakjOEX2nblh334UqWo5b9w==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAxKGDV2PbtbnQuCcSkVyhTLaYnwgyD3vBZOOoFOfSdKIOt80D\nh/Ifre4/s9yXEGSfWDVco29PXj7dzA9tmd1h35w44Bbkcx6VOUT0/ChEseyuVL9r\nNpz4gg/94GkgK1RWafPn+cgqITWWmNKeYuccZEk11krheCt5M3mpz0F7FoD0SeSA\nAEIE/n1PT6FKDsLqVwAilJWiYBhb9MamnJ0ghFMirNDdSJT8KvRI8kfIb+TICmof\nV81GmbTyQ+1Iq4476QnqTVV7kkL/YrjpJlnxlrwR1GIeaHD+D1dHakIdoVBWSumy\n1NZMI6OhuLEI5HCxuZFQ/xo/CTWryjuKSym5OQ+B3nykpnQJtmsGmSQC4XE5llYL\nBR+xRyiozGykRnrRRlULVGepENfREmJfLzjXPLg7xLrGxsxef5mSS7qlaqEC/sYh\npipaOKZ4mIHtpp9uZdnsD3dBabx+HKc/PT/4Q1Y0iIoHPLQ2n7zHfNjHlNSbgxE3\nN6FdvZSfpoijRCq9HYA+KjiHQMNKTsdi9e1IDbEo9UMWq1GHd31Mou7dSGGyRTMH\nTz3KMyJiXo4MtgNVidJUldOqWekufNJEZc7ozMCTz8WLHAOtaHzdBPwi7AmkE50k\nOOqVzRD7kXEW1ABGSucBA4VextBPX7G1XaCtU7bkW2qXbaXVXVVKWOx7sR8CAwEA\nAQKCAgBSYX7XIsG+6epd9RTM7RuxOcFIw9TBcu6MN+tIUvX0Ip969zx2/QSzyOFM\nmGt8FURDXxwAZ1a/3CPt5jrbn4ptdpTHoh+1cYdAlrjzWmWpi8lgPVCMcYiWaV/K\n4RB2xt9rhaJXnrqz7jE5tuhf9IcLJzcUoIvFXqVpYRw5dqw8uetfeAn+0aoo+1Q6\nvw75j0VCDQ7v5J9CloIUVBQjApvIhNI+yEUlrzHVR0K2q0d5RcdfwaV/EkR1in+D\n4Qq0waMtGBffraRXm5VHBcS86isTJZlsu20VAd4XjFAh3zBt0OxOd4oSrPP6bIiJ\nijZ5CxQS9KSUNKlbUO64DGj8466zUsOUXdAc0oXW2cx7k1Uh6tx5ZoAWrErQfJuu\n7aaWhP4n6NY2OnfMiH+IFzn1Om489HvCdGZdHBbzIJ59bufcmeFhIBqbaoI48UqS\nAu/4NpWydMFg2o6AZC5SgUOLp0QaYsguo9FJ/rEMV5g58hehrHrc9G9FR5cpK//8\nAQxHOxOwzU35o51VkQTYk2mpZ+BwhtYvxJ7c6aRO7HWNc6S6WEc4eZ5Oq8ZWon4I\ngYJMVkpKSgwZfU5IeuQ/XZ04GWkBebXIlyqrq7lZVfh+b2O6Yfwo8HKeDiHnXssg\nZ85Z8MftTfmV//V8z1t0Jh9Y4Ax5TIC18p2jsIwes+BOoYqocQKCAQEA6Gp4qkTD\nNQhe/2BpgpnmKqvKiRw7PGkcj9S7QKTSm6bjIoWYZ9l6jXAkUYeh/O31txKvk38y\nSuKoGmtrHfpBakX9eKST6qjLfbsk7ORU+/uA3AfsV7N+7fmlE8HjHZ9XkWRjrHTH\nIPnOtue7xLSF15LygIPV42VfCZ6r4FdrpuXL3Np1QxUrtUe0uWD3eDuf8x6BoLMk\nRJM9yrVe5z4zQQRVYARfAZKZ9XOd9Z+yY9pIS8WGNNkvIjhmBQB2rn2B603EnKL5\nKkws7YLkoHJ50FUBFZpeqwjDZdTx/Ag1AdL41P7LaVCuByBYHFsXcv4ZNIpQPuNc\nrAiPdlDyRzEssQKCAQEA2JVyEbbFqVCgAxGrJTIhCl32z2ApKA1Z5VSuBJsdvsKX\nAhOMgiJviN5o0QulrTcjk+yiYTG4l8Qwq7Bqy+rXJpl5IhSxy84qosZ0hhYazBFF\nvMz6AwthHsbc16+Y4hGZdRUf6Prl8TCoiMjfvFu1nfFXKY6xHBKlKF1JUckZh3Rl\nz0b7UCUXfVHQN5kDV6X5QICooYLarEuTnsrrM/UoPaM7J6zwv0vSZTnyRUfbVdav\npaFkA07QBUcGIyl3o97CoPUAPN7hIPconJQ7OGJgw3zG4dmcQjVg0XFImtVi73Wi\n/Wsyhj7EhFFgmUvoGcQoFN2kAM4eND8DxSqrwezuzwKCAQBXnoQdFM1O+ffuaOd+\nEA0Au0EgzZ7bWHknFDj/1uN7QMBSbHEXfZrBJazbe7mrA/7lZMsCXSqVprtfMm3w\noD73FKnJSKpnBfmpTdtO34nWcWuUB/ym14mbp8j/GNJJgZ/etS0Kf//uQ58Vv2k/\ncBshOUno/0qsbrQUiVgcvHheNDXV7hnVD28p6X2NgavqDPhxDGgsMD0VCTl7aUL9\nBKsH8N1K1HsIub8HFO5RAkKbeFhXEIFW/SXSzdZQPmt/vtsy3V+o+ZvbwVoStI0C\nnAWmb9V30nmsGyKnCk+sB2AMbv/WNLR04LCrCRuOnRzLAlZ4Q0ibXNW8GYIURdIA\n2t4RAoIBAQC3w6VzbjasAGKj9/vUpYHvP+bH8Xk6KInb0UuNUOeeEZD7DAct63Xj\nFYKtzEoYfC5NtnvYJ83Nx4LcSeVze6Btec6ToidxG06Fqgj4NA0slIMtPGXs61zO\nNkDvKGh5eCFSfEgE78zmJjj5Ria/tLiWeE1ST2Xqwc8sPmqVYCGvEo3Hnxxg1ffm\nNuKKKePyHDsuBXwO/e34Pm/UTave++XW1oa8lTn3taegh9gN/HgXjv6ddCBdxiMu\nNyL39pkyizFsTDSVcS5nQA5PPF466w3bECXXP0Ib0xqA2rOV2/QD5TYT+z51NRCc\njEeACi22nv67JYufu0NZZRYX1qrA8f11AoIBACk07TgSk8w8YQMyE1kn2TyXMG/8\nQcauGDFpTkyuITocmZaYjaSr8HvQb4N9myoE4PO6RBRLZYyhT6dzoBH9f03iEJmE\n1IRY85t92QcozRrUO7vZVFkoiHuqft5VmNLSiv+zJFfnTe03UrHrK8cQJXceQxil\nESOmiW4GDWhFh29MwwSoDJjiIEq6UTmmnw0NW9zYbKhbs0PGK9jZRXMU7MBG4JZP\nu+GVTphNKKcqoG+YcVKkRNpVi45vOlJiElqMgLWu4XstAPm1btSYJLD9AvkxfLD7\nkEmk5HYrvma8plK9+4ts/ffWBtDH9eFISJg/LqAd2pQ3cUtNbtQnqvkGhwY=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.874+0000 7ff7c7c94640 1 Processor -- start 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.874+0000 7ff7c7c94640 1 -- start start 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c7c94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c7c94640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0109a60 con 0x7ff7c007a440 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c5a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:35.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c5a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57018/0 (socket says 172.21.15.49:57018) 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c5a30640 1 -- 172.21.15.49:0/2880010366 learned_addr learned my addr 172.21.15.49:0/2880010366 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.875+0000 7ff7c5a30640 1 -- 172.21.15.49:0/2880010366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c0109be0 con 0x7ff7c007a440 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.876+0000 7ff7c5a30640 1 --2- 172.21.15.49:0/2880010366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009820 tx=0x7ff7bc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6abcd0297c1648 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.876+0000 7ff7c4a2e640 1 -- 172.21.15.49:0/2880010366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7bc037070 con 0x7ff7c007a440 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.876+0000 7ff7c4a2e640 1 -- 172.21.15.49:0/2880010366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff7bc0328c0 con 0x7ff7c007a440 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.876+0000 7ff7c4a2e640 1 -- 172.21.15.49:0/2880010366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7bc0306d0 con 0x7ff7c007a440 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2880010366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 msgr2=0x7ff7c0109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2880010366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009820 tx=0x7ff7bc02a300 comp rx=0 tx=0).stop 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2880010366 shutdown_connections 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2880010366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c0109520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2880010366 >> 172.21.15.49:0/2880010366 conn(0x7ff7c01011b0 msgr2=0x7ff7c01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:35.267 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2880010366 shutdown_connections 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2880010366 wait complete. 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 Processor -- start 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 -- start start 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.877+0000 7ff7c7c94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c7c94640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010ace0 con 0x7ff7c007a440 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c5a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c5a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57032/0 (socket says 172.21.15.49:57032) 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c5a30640 1 -- 172.21.15.49:0/2656927572 learned_addr learned my addr 172.21.15.49:0/2656927572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c5a30640 1 -- 172.21.15.49:0/2656927572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c019adb0 con 0x7ff7c007a440 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c5a30640 1 --2- 172.21.15.49:0/2656927572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc032b90 tx=0x7ff7bc032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:35.268 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7bc037040 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff7bc040070 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c019b040 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7bc03be30 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.878+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c019ba00 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.879+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 51743+0+0 (secure 0 0 0) 0x7ff7bc03a470 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.879+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0107510 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.879+0000 7ff7b6ffd640 1 --2- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 0x7ff7ac041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.880+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7ff7bc074230 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.880+0000 7ff7c522f640 1 --2- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 0x7ff7ac041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.880+0000 7ff7c522f640 1 --2- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 0x7ff7ac041be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00098c0 tx=0x7ff7b0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:33.884+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff7bc0313a0 con 0x7ff7c007a440 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.083+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}) -- 0x7ff7c01022f0 con 0x7ff7ac03f720 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.669+0000 7ff7b6ffd640 1 -- 172.21.15.49:0/2656927572 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5136 (secure 0 0 0) 0x7ff7c01022f0 con 0x7ff7ac03f720 2024-09-17T11:37:35.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 msgr2=0x7ff7ac041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 0x7ff7ac041be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00098c0 tx=0x7ff7b0006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 msgr2=0x7ff7c019a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2656927572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc032b90 tx=0x7ff7bc032790 comp rx=0 tx=0).stop 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 shutdown_connections 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2656927572 >> v2:172.21.15.49:6800/1112797402 conn(0x7ff7ac03f720 0x7ff7ac041be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 --2- 172.21.15.49:0/2656927572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7c007a440 0x7ff7c019a850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 >> 172.21.15.49:0/2656927572 conn(0x7ff7c01011b0 msgr2=0x7ff7c0101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 shutdown_connections 2024-09-17T11:37:35.270 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:34.671+0000 7ff7c7c94640 1 -- 172.21.15.49:0/2656927572 wait complete. 2024-09-17T11:37:36.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-17T11:37:36.473 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.682+0000 7f61c9d12640 1 Processor -- start 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.682+0000 7f61c9d12640 1 -- start start 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.682+0000 7f61c9d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.682+0000 7f61c9d12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4107350 con 0x7f61c4106960 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.683+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.683+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57038/0 (socket says 172.21.15.49:57038) 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.683+0000 7f61c37fe640 1 -- 172.21.15.49:0/1057518375 learned_addr learned my addr 172.21.15.49:0/1057518375 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.683+0000 7f61c37fe640 1 -- 172.21.15.49:0/1057518375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c41074d0 con 0x7f61c4106960 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c37fe640 1 --2- 172.21.15.49:0/1057518375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009820 tx=0x7f61b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1449f0fdb3deda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1057518375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b4037070 con 0x7f61c4106960 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1057518375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f61b40328c0 con 0x7f61c4106960 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1057518375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b4030640 con 0x7f61c4106960 2024-09-17T11:37:36.474 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c9d12640 1 -- 172.21.15.49:0/1057518375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 msgr2=0x7f61c4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.684+0000 7f61c9d12640 1 --2- 172.21.15.49:0/1057518375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009820 tx=0x7f61b402a270 comp rx=0 tx=0).stop 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 -- 172.21.15.49:0/1057518375 shutdown_connections 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 --2- 172.21.15.49:0/1057518375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c4106d60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 -- 172.21.15.49:0/1057518375 >> 172.21.15.49:0/1057518375 conn(0x7f61c4102250 msgr2=0x7f61c4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 -- 172.21.15.49:0/1057518375 shutdown_connections 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 -- 172.21.15.49:0/1057518375 wait complete. 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 Processor -- start 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.685+0000 7f61c9d12640 1 -- start start 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c9d12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c9d12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410ade0 con 0x7f61c4106960 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57042/0 (socket says 172.21.15.49:57042) 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c37fe640 1 -- 172.21.15.49:0/3077381428 learned_addr learned my addr 172.21.15.49:0/3077381428 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:36.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c37fe640 1 -- 172.21.15.49:0/3077381428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c419ada0 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c37fe640 1 --2- 172.21.15.49:0/3077381428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000c00 tx=0x7f61b4030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b4037040 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.686+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f61b4040070 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.687+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c419b030 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.687+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b403a390 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.687+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c419b9f0 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.687+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 51743+0+0 (secure 0 0 0) 0x7f61b403a610 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.688+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c4106d60 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.688+0000 7f61c17fa640 1 --2- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 0x7f61b0041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.688+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f61b4074440 con 0x7f61c4106960 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.688+0000 7f61baffd640 1 --2- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 0x7f61b0041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.688+0000 7f61baffd640 1 --2- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 0x7f61b0041be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098c0 tx=0x7f61ac006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:36.476 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.693+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f61b40313a0 con 0x7f61c4106960 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.901+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}) -- 0x7f61c41033a0 con 0x7f61b003f720 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.904+0000 7f61c17fa640 1 -- 172.21.15.49:0/3077381428 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61c41033a0 con 0x7f61b003f720 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 msgr2=0x7f61b0041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 --2- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 0x7f61b0041be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098c0 tx=0x7f61ac006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 msgr2=0x7f61c419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 --2- 172.21.15.49:0/3077381428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000c00 tx=0x7f61b4030b70 comp rx=0 tx=0).stop 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 shutdown_connections 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 --2- 172.21.15.49:0/3077381428 >> v2:172.21.15.49:6800/1112797402 conn(0x7f61b003f720 0x7f61b0041be0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 --2- 172.21.15.49:0/3077381428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c4106960 0x7f61c419a840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 >> 172.21.15.49:0/3077381428 conn(0x7f61c4102250 msgr2=0x7f61c4102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 shutdown_connections 2024-09-17T11:37:36.477 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:35.906+0000 7f61c9d12640 1 -- 172.21.15.49:0/3077381428 wait complete. 2024-09-17T11:37:36.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:36.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:37.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:37 smithi049 ceph-mon[30017]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:37.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:37 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:37.857 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.883+0000 7f2e79538640 1 Processor -- start 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e79538640 1 -- start start 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e79538640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e79538640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74109a60 con 0x7f2e7407a440 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e72ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e72ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57056/0 (socket says 172.21.15.49:57056) 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e72ffd640 1 -- 172.21.15.49:0/393496086 learned_addr learned my addr 172.21.15.49:0/393496086 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.884+0000 7f2e72ffd640 1 -- 172.21.15.49:0/393496086 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e74109be0 con 0x7f2e7407a440 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.885+0000 7f2e72ffd640 1 --2- 172.21.15.49:0/393496086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009820 tx=0x7f2e6402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5bd38f5bd168ec server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.885+0000 7f2e71ffb640 1 -- 172.21.15.49:0/393496086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64037070 con 0x7f2e7407a440 2024-09-17T11:37:37.858 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.885+0000 7f2e71ffb640 1 -- 172.21.15.49:0/393496086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f2e640328c0 con 0x7f2e7407a440 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.885+0000 7f2e71ffb640 1 -- 172.21.15.49:0/393496086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e640306d0 con 0x7f2e7407a440 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 -- 172.21.15.49:0/393496086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 msgr2=0x7f2e74109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 --2- 172.21.15.49:0/393496086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009820 tx=0x7f2e6402a300 comp rx=0 tx=0).stop 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 -- 172.21.15.49:0/393496086 shutdown_connections 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 --2- 172.21.15.49:0/393496086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e74109520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 -- 172.21.15.49:0/393496086 >> 172.21.15.49:0/393496086 conn(0x7f2e741011b0 msgr2=0x7f2e741035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 -- 172.21.15.49:0/393496086 shutdown_connections 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.886+0000 7f2e79538640 1 -- 172.21.15.49:0/393496086 wait complete. 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e79538640 1 Processor -- start 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e79538640 1 -- start start 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e79538640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e79538640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7410ace0 con 0x7f2e7407a440 2024-09-17T11:37:37.859 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e72ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e72ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57060/0 (socket says 172.21.15.49:57060) 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e72ffd640 1 -- 172.21.15.49:0/2290265319 learned_addr learned my addr 172.21.15.49:0/2290265319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e72ffd640 1 -- 172.21.15.49:0/2290265319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7419adb0 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.887+0000 7f2e72ffd640 1 --2- 172.21.15.49:0/2290265319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000c00 tx=0x7f2e64030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.888+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64037040 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.888+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f2e64040070 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.888+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7419b040 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.888+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e6403be20 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.888+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7419ba00 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.889+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 51743+0+0 (secure 0 0 0) 0x7f2e6403b430 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.889+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e74107510 con 0x7f2e7407a440 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.889+0000 7f2e5bfff640 1 --2- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 0x7f2e5c041be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.889+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f2e64074270 con 0x7f2e7407a440 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.889+0000 7f2e727fc640 1 --2- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 0x7f2e5c041be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.890+0000 7f2e727fc640 1 --2- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 0x7f2e5c041be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098c0 tx=0x7f2e68006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:36.894+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f2e64043360 con 0x7f2e7407a440 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.099+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}) -- 0x7f2e741022f0 con 0x7f2e5c03f720 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.103+0000 7f2e5bfff640 1 -- 172.21.15.49:0/2290265319 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2e741022f0 con 0x7f2e5c03f720 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 msgr2=0x7f2e5c041be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 --2- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 0x7f2e5c041be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098c0 tx=0x7f2e68006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 msgr2=0x7f2e7419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 --2- 172.21.15.49:0/2290265319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000c00 tx=0x7f2e64030a60 comp rx=0 tx=0).stop 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 shutdown_connections 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 --2- 172.21.15.49:0/2290265319 >> v2:172.21.15.49:6800/1112797402 conn(0x7f2e5c03f720 0x7f2e5c041be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 --2- 172.21.15.49:0/2290265319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e7407a440 0x7f2e7419a850 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:37.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.105+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 >> 172.21.15.49:0/2290265319 conn(0x7f2e741011b0 msgr2=0x7f2e74101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:37.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.106+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 shutdown_connections 2024-09-17T11:37:37.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:37.106+0000 7f2e79538640 1 -- 172.21.15.49:0/2290265319 wait complete. 2024-09-17T11:37:37.862 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-09-17T11:37:38.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:38 smithi049 ceph-mon[30017]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:39 smithi049 ceph-mon[30017]: mgrmap e13: y(active, since 6s) 2024-09-17T11:37:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:39 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$C.PrTi9EeBfVjHcssFPDqOwlywoF1Ym6Yej9aip5oQ8NaOdRM.WWC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726573059, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe904610640 1 Processor -- start 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe904610640 1 -- start start 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe904610640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe904610640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc106270 con 0x7fe8fc105880 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe9023ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe9023ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57068/0 (socket says 172.21.15.49:57068) 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.641+0000 7fe9023ac640 1 -- 172.21.15.49:0/1351744325 learned_addr learned my addr 172.21.15.49:0/1351744325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.642+0000 7fe9023ac640 1 -- 172.21.15.49:0/1351744325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc1063f0 con 0x7fe8fc105880 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.642+0000 7fe9023ac640 1 --2- 172.21.15.49:0/1351744325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009e80 tx=0x7fe8ec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65e2777456c053d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.642+0000 7fe9013aa640 1 -- 172.21.15.49:0/1351744325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec036070 con 0x7fe8fc105880 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.642+0000 7fe9013aa640 1 -- 172.21.15.49:0/1351744325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fe8ec031880 con 0x7fe8fc105880 2024-09-17T11:37:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.642+0000 7fe9013aa640 1 -- 172.21.15.49:0/1351744325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec007880 con 0x7fe8fc105880 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 -- 172.21.15.49:0/1351744325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 msgr2=0x7fe8fc105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 --2- 172.21.15.49:0/1351744325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009e80 tx=0x7fe8ec02a7a0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 -- 172.21.15.49:0/1351744325 shutdown_connections 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 --2- 172.21.15.49:0/1351744325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc105c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 -- 172.21.15.49:0/1351744325 >> 172.21.15.49:0/1351744325 conn(0x7fe8fc101170 msgr2=0x7fe8fc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 -- 172.21.15.49:0/1351744325 shutdown_connections 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.643+0000 7fe904610640 1 -- 172.21.15.49:0/1351744325 wait complete. 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe904610640 1 Processor -- start 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe904610640 1 -- start start 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe904610640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe904610640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc109d00 con 0x7fe8fc105880 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe9023ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe9023ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57082/0 (socket says 172.21.15.49:57082) 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe9023ac640 1 -- 172.21.15.49:0/1944061473 learned_addr learned my addr 172.21.15.49:0/1944061473 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.644+0000 7fe9023ac640 1 -- 172.21.15.49:0/1944061473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc19aeb0 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe9023ac640 1 --2- 172.21.15.49:0/1944061473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec008dc0 tx=0x7fe8ec008df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec041020 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fe8ec03c8f0 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc19b140 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec036040 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.645+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc19bb80 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.646+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7fe8ec044020 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.646+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc105d00 con 0x7fe8fc105880 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.646+0000 7fe8fb7fe640 1 --2- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 0x7fe8e4041c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.646+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7fe8ec0755e0 con 0x7fe8fc105880 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.646+0000 7fe901bab640 1 --2- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 0x7fe8e4041c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.647+0000 7fe901bab640 1 --2- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 0x7fe8e4041c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098c0 tx=0x7fe8f0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.651+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe8ec04b020 con 0x7fe8fc105880 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:38.849+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe8fc19c3d0 con 0x7fe8e403f7c0 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.097+0000 7fe8fb7fe640 1 -- 172.21.15.49:0/1944061473 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe8fc19c3d0 con 0x7fe8e403f7c0 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.099+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 msgr2=0x7fe8e4041c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.099+0000 7fe904610640 1 --2- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 0x7fe8e4041c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098c0 tx=0x7fe8f0006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.099+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 msgr2=0x7fe8fc19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.099+0000 7fe904610640 1 --2- 172.21.15.49:0/1944061473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec008dc0 tx=0x7fe8ec008df0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 shutdown_connections 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 --2- 172.21.15.49:0/1944061473 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe8e403f7c0 0x7fe8e4041c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 --2- 172.21.15.49:0/1944061473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8fc105880 0x7fe8fc19a950 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 >> 172.21.15.49:0/1944061473 conn(0x7fe8fc101170 msgr2=0x7fe8fc07a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 shutdown_connections 2024-09-17T11:37:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:39.100+0000 7fe904610640 1 -- 172.21.15.49:0/1944061473 wait complete. 2024-09-17T11:37:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-09-17T11:37:40.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:40 smithi049 ceph-mon[30017]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:40.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756956273' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-17T11:37:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T11:37:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f154b084640 1 Processor -- start 2024-09-17T11:37:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f154b084640 1 -- start start 2024-09-17T11:37:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f154b084640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f154b084640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1544109b40 con 0x7f154407a360 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f1548e20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f1548e20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57084/0 (socket says 172.21.15.49:57084) 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.002+0000 7f1548e20640 1 -- 172.21.15.49:0/755766332 learned_addr learned my addr 172.21.15.49:0/755766332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.003+0000 7f1548e20640 1 -- 172.21.15.49:0/755766332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1544109cc0 con 0x7f154407a360 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.003+0000 7f1548e20640 1 --2- 172.21.15.49:0/755766332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1538009820 tx=0x7f153802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563a9865e8ed9e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f15437fe640 1 -- 172.21.15.49:0/755766332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538037070 con 0x7f154407a360 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f15437fe640 1 -- 172.21.15.49:0/755766332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f15380328c0 con 0x7f154407a360 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f15437fe640 1 -- 172.21.15.49:0/755766332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538030640 con 0x7f154407a360 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 -- 172.21.15.49:0/755766332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 msgr2=0x7f1544109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 --2- 172.21.15.49:0/755766332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1538009820 tx=0x7f153802a270 comp rx=0 tx=0).stop 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 -- 172.21.15.49:0/755766332 shutdown_connections 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 --2- 172.21.15.49:0/755766332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f1544109600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 -- 172.21.15.49:0/755766332 >> 172.21.15.49:0/755766332 conn(0x7f15441011b0 msgr2=0x7f15441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 -- 172.21.15.49:0/755766332 shutdown_connections 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.004+0000 7f154b084640 1 -- 172.21.15.49:0/755766332 wait complete. 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f154b084640 1 Processor -- start 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f154b084640 1 -- start start 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f154b084640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f154b084640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f154410ad00 con 0x7f154407a360 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f1548e20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f1548e20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57092/0 (socket says 172.21.15.49:57092) 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f1548e20640 1 -- 172.21.15.49:0/2756956273 learned_addr learned my addr 172.21.15.49:0/2756956273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.005+0000 7f1548e20640 1 -- 172.21.15.49:0/2756956273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154419ad80 con 0x7f154407a360 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f1548e20640 1 --2- 172.21.15.49:0/2756956273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538037040 con 0x7f154407a360 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f1538040070 con 0x7f154407a360 2024-09-17T11:37:40.742 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154419b010 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153803a390 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.006+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154419b9d0 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.007+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f153803b430 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.007+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15441075f0 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.007+0000 7f1541ffb640 1 --2- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 0x7f1534041c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.007+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f15380743a0 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.007+0000 7f1543fff640 1 --2- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 0x7f1534041c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.008+0000 7f1543fff640 1 --2- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 0x7f1534041c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f1530006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.012+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f15380327b0 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.201+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f154419c110 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.201+0000 7f1541ffb640 1 -- 172.21.15.49:0/2756956273 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1538033ce0 con 0x7f154407a360 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 msgr2=0x7f1534041c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:40.743 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 --2- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 0x7f1534041c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f1530006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 msgr2=0x7f154419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 --2- 172.21.15.49:0/2756956273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538030b70 comp rx=0 tx=0).stop 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 shutdown_connections 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 --2- 172.21.15.49:0/2756956273 >> v2:172.21.15.49:6800/1112797402 conn(0x7f153403f7c0 0x7f1534041c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 --2- 172.21.15.49:0/2756956273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f154407a360 0x7f154419a820 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 >> 172.21.15.49:0/2756956273 conn(0x7f15441011b0 msgr2=0x7f1544101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 shutdown_connections 2024-09-17T11:37:40.744 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:40.204+0000 7f154b084640 1 -- 172.21.15.49:0/2756956273 wait complete. 2024-09-17T11:37:40.748 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T11:37:40.748 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-09-17T11:37:40.756 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T11:37:40.756 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-09-17T11:37:40.756 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-09-17T11:37:40.756 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: Password: t9fctq2msj 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:40.758 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config directory 2024-09-17T11:37:42.039 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.166+0000 7fb21a078640 1 Processor -- start 2024-09-17T11:37:42.039 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.166+0000 7fb21a078640 1 -- start start 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.166+0000 7fb21a078640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.166+0000 7fb21a078640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb214106250 con 0x7fb214105860 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.167+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.167+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57094/0 (socket says 172.21.15.49:57094) 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.167+0000 7fb2137fe640 1 -- 172.21.15.49:0/2624218957 learned_addr learned my addr 172.21.15.49:0/2624218957 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.167+0000 7fb2137fe640 1 -- 172.21.15.49:0/2624218957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2141063d0 con 0x7fb214105860 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb2137fe640 1 --2- 172.21.15.49:0/2624218957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb204009820 tx=0x7fb20402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121f07531dba3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb2127fc640 1 -- 172.21.15.49:0/2624218957 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb204037070 con 0x7fb214105860 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb2127fc640 1 -- 172.21.15.49:0/2624218957 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fb2040328c0 con 0x7fb214105860 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb2127fc640 1 -- 172.21.15.49:0/2624218957 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb204030640 con 0x7fb214105860 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb21a078640 1 -- 172.21.15.49:0/2624218957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 msgr2=0x7fb214105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb21a078640 1 --2- 172.21.15.49:0/2624218957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb204009820 tx=0x7fb20402a270 comp rx=0 tx=0).stop 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb21a078640 1 -- 172.21.15.49:0/2624218957 shutdown_connections 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb21a078640 1 --2- 172.21.15.49:0/2624218957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb214105c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.168+0000 7fb21a078640 1 -- 172.21.15.49:0/2624218957 >> 172.21.15.49:0/2624218957 conn(0x7fb2141011b0 msgr2=0x7fb2141035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 -- 172.21.15.49:0/2624218957 shutdown_connections 2024-09-17T11:37:42.041 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 -- 172.21.15.49:0/2624218957 wait complete. 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 Processor -- start 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 -- start start 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.169+0000 7fb21a078640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb214109ce0 con 0x7fb214105860 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57096/0 (socket says 172.21.15.49:57096) 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb2137fe640 1 -- 172.21.15.49:0/435167170 learned_addr learned my addr 172.21.15.49:0/435167170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb2137fe640 1 -- 172.21.15.49:0/435167170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb214190d50 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb2137fe640 1 --2- 172.21.15.49:0/435167170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb20402a830 tx=0x7fb204030bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb204037040 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fb204040070 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21418f140 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb20403be40 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.170+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21418f6c0 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.171+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7fb20403a4a0 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.171+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb214105c60 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.171+0000 7fb210ff9640 1 --2- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 0x7fb200041cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.171+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7fb204074390 con 0x7fb214105860 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.172+0000 7fb212ffd640 1 --2- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 0x7fb200041cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.172+0000 7fb212ffd640 1 --2- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 0x7fb200041cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098c0 tx=0x7fb1fc006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.176+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb204039950 con 0x7fb214105860 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.462+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb21418fbf0 con 0x7fb214105860 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.464+0000 7fb210ff9640 1 -- 172.21.15.49:0/435167170 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb204030d50 con 0x7fb214105860 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 msgr2=0x7fb200041cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 --2- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 0x7fb200041cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098c0 tx=0x7fb1fc006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 msgr2=0x7fb2141907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 --2- 172.21.15.49:0/435167170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb20402a830 tx=0x7fb204030bc0 comp rx=0 tx=0).stop 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 shutdown_connections 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 --2- 172.21.15.49:0/435167170 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb20003f810 0x7fb200041cd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 --2- 172.21.15.49:0/435167170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb214105860 0x7fb2141907f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 >> 172.21.15.49:0/435167170 conn(0x7fb2141011b0 msgr2=0x7fb214079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 shutdown_connections 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-09-17T11:37:41.467+0000 7fb21a078640 1 -- 172.21.15.49:0/435167170 wait complete. 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /sbin/cephadm shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /sbin/cephadm shell 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T11:37:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:42.046 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-09-17T11:37:42.077 INFO:tasks.cephadm:Fetching config... 2024-09-17T11:37:42.077 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:37:42.078 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T11:37:42.095 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T11:37:42.095 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:37:42.095 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T11:37:42.151 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T11:37:42.151 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:37:42.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T11:37:42.217 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T11:37:42.217 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:37:42.217 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T11:37:42.273 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T11:37:42.273 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvqi/fE4zhweHoTePDu9d4I6OKhDKDeNmkh+RfUMkrMjRumh2hgh3fLIvJN125Fimtc6lw9i/6X2FSK8FjTedG/gGbxA91t0/TP2zu+2hN7g86rWFNqPdYnNkRtQ8IkvlHz0AFjB3Hqy92UuU+vFmXXtxnf3UcfCG/z6NhW1wICmg5fPmtbjBTv0DalSuixN0SlNLDhOFsfASle7jUwBAqFCd+eJBl7Kf6q+XYSkwMn3tqNyWEYY7tMApZOp69tb+y5rKndHJGMlnchUH8/ZwanNE0kgguxTmh18UXRFWz3AydMyLD7Gb6ujHDcVzYuCzzLCETTzFGY0x46EkBgdY+odTzTxG6LD6KtxUL04C7B/Nhr8P88ICpzESaYcdSXK5+XndIaogDNz0OXo/aOkEdMsxAXTPPYAA00ZLiLaEQfY9U5UyElRV5I46tRv3/h12LZZZ8vn2fKMPUOkt/qKhbInCV1wNcpTu9hd521rK8Pl0idcibn6B4hYLC6L36hcs= ceph-f97d9e96-74e8-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T11:37:42.349 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvqi/fE4zhweHoTePDu9d4I6OKhDKDeNmkh+RfUMkrMjRumh2hgh3fLIvJN125Fimtc6lw9i/6X2FSK8FjTedG/gGbxA91t0/TP2zu+2hN7g86rWFNqPdYnNkRtQ8IkvlHz0AFjB3Hqy92UuU+vFmXXtxnf3UcfCG/z6NhW1wICmg5fPmtbjBTv0DalSuixN0SlNLDhOFsfASle7jUwBAqFCd+eJBl7Kf6q+XYSkwMn3tqNyWEYY7tMApZOp69tb+y5rKndHJGMlnchUH8/ZwanNE0kgguxTmh18UXRFWz3AydMyLD7Gb6ujHDcVzYuCzzLCETTzFGY0x46EkBgdY+odTzTxG6LD6KtxUL04C7B/Nhr8P88ICpzESaYcdSXK5+XndIaogDNz0OXo/aOkEdMsxAXTPPYAA00ZLiLaEQfY9U5UyElRV5I46tRv3/h12LZZZ8vn2fKMPUOkt/qKhbInCV1wNcpTu9hd521rK8Pl0idcibn6B4hYLC6L36hcs= ceph-f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:37:42.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvqi/fE4zhweHoTePDu9d4I6OKhDKDeNmkh+RfUMkrMjRumh2hgh3fLIvJN125Fimtc6lw9i/6X2FSK8FjTedG/gGbxA91t0/TP2zu+2hN7g86rWFNqPdYnNkRtQ8IkvlHz0AFjB3Hqy92UuU+vFmXXtxnf3UcfCG/z6NhW1wICmg5fPmtbjBTv0DalSuixN0SlNLDhOFsfASle7jUwBAqFCd+eJBl7Kf6q+XYSkwMn3tqNyWEYY7tMApZOp69tb+y5rKndHJGMlnchUH8/ZwanNE0kgguxTmh18UXRFWz3AydMyLD7Gb6ujHDcVzYuCzzLCETTzFGY0x46EkBgdY+odTzTxG6LD6KtxUL04C7B/Nhr8P88ICpzESaYcdSXK5+XndIaogDNz0OXo/aOkEdMsxAXTPPYAA00ZLiLaEQfY9U5UyElRV5I46tRv3/h12LZZZ8vn2fKMPUOkt/qKhbInCV1wNcpTu9hd521rK8Pl0idcibn6B4hYLC6L36hcs= ceph-f97d9e96-74e8-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T11:37:42.419 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvqi/fE4zhweHoTePDu9d4I6OKhDKDeNmkh+RfUMkrMjRumh2hgh3fLIvJN125Fimtc6lw9i/6X2FSK8FjTedG/gGbxA91t0/TP2zu+2hN7g86rWFNqPdYnNkRtQ8IkvlHz0AFjB3Hqy92UuU+vFmXXtxnf3UcfCG/z6NhW1wICmg5fPmtbjBTv0DalSuixN0SlNLDhOFsfASle7jUwBAqFCd+eJBl7Kf6q+XYSkwMn3tqNyWEYY7tMApZOp69tb+y5rKndHJGMlnchUH8/ZwanNE0kgguxTmh18UXRFWz3AydMyLD7Gb6ujHDcVzYuCzzLCETTzFGY0x46EkBgdY+odTzTxG6LD6KtxUL04C7B/Nhr8P88ICpzESaYcdSXK5+XndIaogDNz0OXo/aOkEdMsxAXTPPYAA00ZLiLaEQfY9U5UyElRV5I46tRv3/h12LZZZ8vn2fKMPUOkt/qKhbInCV1wNcpTu9hd521rK8Pl0idcibn6B4hYLC6L36hcs= ceph-f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:37:42.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T11:37:42.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/435167170' entity='client.admin' 2024-09-17T11:37:42.686 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:37:43.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 -- 172.21.15.49:0/2189851096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 msgr2=0x7f29881016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:43.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 --2- 172.21.15.49:0/2189851096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f29881016e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2984009880 tx=0x7f298402a5d0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 -- 172.21.15.49:0/2189851096 shutdown_connections 2024-09-17T11:37:43.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 --2- 172.21.15.49:0/2189851096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f29881016e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 -- 172.21.15.49:0/2189851096 >> 172.21.15.49:0/2189851096 conn(0x7f29880fcbd0 msgr2=0x7f29880feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:43.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 -- 172.21.15.49:0/2189851096 shutdown_connections 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.165+0000 7f298fbd3640 1 -- 172.21.15.49:0/2189851096 wait complete. 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298fbd3640 1 Processor -- start 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298fbd3640 1 -- start start 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298fbd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298fbd3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29881069f0 con 0x7f29881012e0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298d96f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298d96f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:57116/0 (socket says 172.21.15.49:57116) 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.166+0000 7f298d96f640 1 -- 172.21.15.49:0/1423758864 learned_addr learned my addr 172.21.15.49:0/1423758864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f298d96f640 1 -- 172.21.15.49:0/1423758864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881967e0 con 0x7f29881012e0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f298d96f640 1 --2- 172.21.15.49:0/1423758864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29840057c0 tx=0x7f29840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f298403f070 con 0x7f29881012e0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f2984003ba0 con 0x7f29881012e0 2024-09-17T11:37:43.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984035030 con 0x7f29881012e0 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2988196a70 con 0x7f29881012e0 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.167+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2988197430 con 0x7f29881012e0 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.168+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f2984042070 con 0x7f29881012e0 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.168+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2988107170 con 0x7f29881012e0 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.168+0000 7f297effd640 1 --2- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 0x7f2974041c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.168+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f2984074b40 con 0x7f29881012e0 2024-09-17T11:37:43.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.168+0000 7f298d16e640 1 --2- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 0x7f2974041c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:43.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.169+0000 7f298d16e640 1 --2- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 0x7f2974041c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f29780098c0 tx=0x7f2978006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.173+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f2984037d90 con 0x7f29881012e0 2024-09-17T11:37:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.359+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2988197a00 con 0x7f29881012e0 2024-09-17T11:37:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.361+0000 7f297effd640 1 -- 172.21.15.49:0/1423758864 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f298403b460 con 0x7f29881012e0 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 msgr2=0x7f2974041c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 --2- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 0x7f2974041c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f29780098c0 tx=0x7f2978006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 msgr2=0x7f2988196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 --2- 172.21.15.49:0/1423758864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29840057c0 tx=0x7f29840048f0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 shutdown_connections 2024-09-17T11:37:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 --2- 172.21.15.49:0/1423758864 >> v2:172.21.15.49:6800/1112797402 conn(0x7f297403f750 0x7f2974041c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 --2- 172.21.15.49:0/1423758864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29881012e0 0x7f2988196280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 >> 172.21.15.49:0/1423758864 conn(0x7f29880fcbd0 msgr2=0x7f29880fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 shutdown_connections 2024-09-17T11:37:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:43.364+0000 7f298fbd3640 1 -- 172.21.15.49:0/1423758864 wait complete. 2024-09-17T11:37:44.302 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T11:37:44.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T11:37:44.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1423758864' entity='client.admin' 2024-09-17T11:37:44.746 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:37:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 -- 172.21.15.49:0/4215016576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 msgr2=0x7f406c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 --2- 172.21.15.49:0/4215016576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c1016f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f405c009880 tx=0x7f405c02a5d0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 -- 172.21.15.49:0/4215016576 shutdown_connections 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 --2- 172.21.15.49:0/4215016576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c1016f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 -- 172.21.15.49:0/4215016576 >> 172.21.15.49:0/4215016576 conn(0x7f406c0fcbc0 msgr2=0x7f406c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 -- 172.21.15.49:0/4215016576 shutdown_connections 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.225+0000 7f4070baf640 1 -- 172.21.15.49:0/4215016576 wait complete. 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f4070baf640 1 Processor -- start 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f4070baf640 1 -- start start 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f4070baf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f4070baf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f406c106a00 con 0x7f406c1012f0 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f406a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f406a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51222/0 (socket says 172.21.15.49:51222) 2024-09-17T11:37:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.226+0000 7f406a59c640 1 -- 172.21.15.49:0/3552425903 learned_addr learned my addr 172.21.15.49:0/3552425903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f406a59c640 1 -- 172.21.15.49:0/3552425903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0fedc0 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f406a59c640 1 --2- 172.21.15.49:0/3552425903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f405c0057c0 tx=0x7f405c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c03f070 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f405c003ba0 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f405c035030 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c0fd1b0 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.227+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c0fd730 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.228+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f405c042070 con 0x7f406c1012f0 2024-09-17T11:37:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.228+0000 7f405b7fe640 1 --2- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 0x7f4054041c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.228+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c107180 con 0x7f406c1012f0 2024-09-17T11:37:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.229+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f405c074b70 con 0x7f406c1012f0 2024-09-17T11:37:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.229+0000 7f4069d9b640 1 --2- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 0x7f4054041c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.229+0000 7f4069d9b640 1 --2- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 0x7f4054041c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40600098c0 tx=0x7f4060006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.236+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f405c036260 con 0x7f406c1012f0 2024-09-17T11:37:45.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.457+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f406c0fdaa0 con 0x7f405403f750 2024-09-17T11:37:45.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.460+0000 7f405b7fe640 1 -- 172.21.15.49:0/3552425903 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f406c0fdaa0 con 0x7f405403f750 2024-09-17T11:37:45.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 msgr2=0x7f4054041c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 --2- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 0x7f4054041c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40600098c0 tx=0x7f4060006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 msgr2=0x7f406c0fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 --2- 172.21.15.49:0/3552425903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f405c0057c0 tx=0x7f405c0048f0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 shutdown_connections 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 --2- 172.21.15.49:0/3552425903 >> v2:172.21.15.49:6800/1112797402 conn(0x7f405403f750 0x7f4054041c10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 --2- 172.21.15.49:0/3552425903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1012f0 0x7f406c0fe860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 >> 172.21.15.49:0/3552425903 conn(0x7f406c0fcbc0 msgr2=0x7f406c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 shutdown_connections 2024-09-17T11:37:45.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:45.464+0000 7f4070baf640 1 -- 172.21.15.49:0/3552425903 wait complete. 2024-09-17T11:37:46.051 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-09-17T11:37:46.052 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:37:46.052 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-09-17T11:37:46.073 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:37:46.073 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:37:46.131 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-09-17T11:37:46.132 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch host add smithi129 2024-09-17T11:37:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"} : dispatch 2024-09-17T11:37:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:37:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 -- 172.21.15.49:0/3626904567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 msgr2=0x7fb3380755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/3626904567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb3380755e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb334009880 tx=0x7fb33402a5d0 comp rx=0 tx=0).stop 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 -- 172.21.15.49:0/3626904567 shutdown_connections 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/3626904567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb3380755e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 -- 172.21.15.49:0/3626904567 >> 172.21.15.49:0/3626904567 conn(0x7fb3380fdcd0 msgr2=0x7fb338100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 -- 172.21.15.49:0/3626904567 shutdown_connections 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.038+0000 7fb33fe9a640 1 -- 172.21.15.49:0/3626904567 wait complete. 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33fe9a640 1 Processor -- start 2024-09-17T11:37:47.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33fe9a640 1 -- start start 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33fe9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33fe9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb338105930 con 0x7fb338076f80 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33dc36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33dc36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51252/0 (socket says 172.21.15.49:51252) 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33dc36640 1 -- 172.21.15.49:0/154124190 learned_addr learned my addr 172.21.15.49:0/154124190 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33dc36640 1 -- 172.21.15.49:0/154124190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33819dec0 con 0x7fb338076f80 2024-09-17T11:37:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33dc36640 1 --2- 172.21.15.49:0/154124190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3340057c0 tx=0x7fb3340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb33403f070 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33819e150 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.039+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fb334003ba0 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.040+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33819eb10 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.040+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb334035030 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.040+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7fb334042070 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.041+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb338106070 con 0x7fb338076f80 2024-09-17T11:37:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.041+0000 7fb32effd640 1 --2- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 0x7fb324041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.041+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7fb334074c90 con 0x7fb338076f80 2024-09-17T11:37:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.041+0000 7fb33d435640 1 --2- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 0x7fb324041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.041+0000 7fb33d435640 1 --2- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 0x7fb324041bc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3280098c0 tx=0x7fb328006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.045+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb334036260 con 0x7fb338076f80 2024-09-17T11:37:47.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:47.238+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7fb33819f850 con 0x7fb32403f700 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:37:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:37:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:47 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:48 smithi049 ceph-mon[30017]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:37:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:48 smithi049 ceph-mon[30017]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:37:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:48 smithi049 ceph-mon[30017]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:37:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:48 smithi049 ceph-mon[30017]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:49.899 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:49 smithi049 ceph-mon[30017]: Deploying cephadm binary to smithi129 2024-09-17T11:37:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.082+0000 7fb32effd640 1 -- 172.21.15.49:0/154124190 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb33819f850 con 0x7fb32403f700 2024-09-17T11:37:51.084 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 msgr2=0x7fb324041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 0x7fb324041bc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb3280098c0 tx=0x7fb328006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 msgr2=0x7fb33819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/154124190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb3340057c0 tx=0x7fb3340048f0 comp rx=0 tx=0).stop 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 shutdown_connections 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/154124190 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb32403f700 0x7fb324041bc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 --2- 172.21.15.49:0/154124190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb338076f80 0x7fb33819d960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 >> 172.21.15.49:0/154124190 conn(0x7fb3380fdcd0 msgr2=0x7fb3380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 shutdown_connections 2024-09-17T11:37:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:51.085+0000 7fb33fe9a640 1 -- 172.21.15.49:0/154124190 wait complete. 2024-09-17T11:37:51.715 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T11:37:51.967 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:37:52.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:52.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:52 smithi049 ceph-mon[30017]: Added host smithi129 2024-09-17T11:37:52.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:52.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:52.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:52.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 -- 172.21.15.49:0/1236861073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 msgr2=0x7f97a0108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:52.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 --2- 172.21.15.49:0/1236861073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0108890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f979c009880 tx=0x7f979c02a670 comp rx=0 tx=0).stop 2024-09-17T11:37:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 -- 172.21.15.49:0/1236861073 shutdown_connections 2024-09-17T11:37:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 --2- 172.21.15.49:0/1236861073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0108890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 -- 172.21.15.49:0/1236861073 >> 172.21.15.49:0/1236861073 conn(0x7f97a0078210 msgr2=0x7f97a0078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 -- 172.21.15.49:0/1236861073 shutdown_connections 2024-09-17T11:37:52.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.423+0000 7f97a7d59640 1 -- 172.21.15.49:0/1236861073 wait complete. 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a7d59640 1 Processor -- start 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a7d59640 1 -- start start 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a7d59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a7d59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010c0e0 con 0x7f97a01084b0 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a5af5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a5af5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51268/0 (socket says 172.21.15.49:51268) 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.424+0000 7f97a5af5640 1 -- 172.21.15.49:0/661088991 learned_addr learned my addr 172.21.15.49:0/661088991 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f97a5af5640 1 -- 172.21.15.49:0/661088991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a0197bc0 con 0x7f97a01084b0 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f97a5af5640 1 --2- 172.21.15.49:0/661088991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f979c005ab0 tx=0x7f979c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f979c03f070 con 0x7f97a01084b0 2024-09-17T11:37:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f979c003a70 con 0x7f97a01084b0 2024-09-17T11:37:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a0197eb0 con 0x7f97a01084b0 2024-09-17T11:37:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f979c035030 con 0x7f97a01084b0 2024-09-17T11:37:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.425+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a019cf60 con 0x7f97a01084b0 2024-09-17T11:37:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.426+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f979c042070 con 0x7f97a01084b0 2024-09-17T11:37:52.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.426+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a0103a50 con 0x7f97a01084b0 2024-09-17T11:37:52.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.426+0000 7f9796ffd640 1 --2- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 0x7f978c041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:52.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.426+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7f979c074a70 con 0x7f97a01084b0 2024-09-17T11:37:52.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.426+0000 7f97a52f4640 1 --2- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 0x7f978c041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:52.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.427+0000 7f97a52f4640 1 --2- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 0x7f978c041bc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97900098c0 tx=0x7f9790006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:52.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.431+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f979c0372d0 con 0x7f97a01084b0 2024-09-17T11:37:52.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.620+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97a0108910 con 0x7f978c03f700 2024-09-17T11:37:52.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.621+0000 7f9796ffd640 1 -- 172.21.15.49:0/661088991 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f97a0108910 con 0x7f978c03f700 2024-09-17T11:37:52.623 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:37:52.623 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 msgr2=0x7f978c041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 --2- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 0x7f978c041bc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97900098c0 tx=0x7f9790006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 msgr2=0x7f97a0197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 --2- 172.21.15.49:0/661088991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f979c005ab0 tx=0x7f979c0048f0 comp rx=0 tx=0).stop 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 shutdown_connections 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 --2- 172.21.15.49:0/661088991 >> v2:172.21.15.49:6800/1112797402 conn(0x7f978c03f700 0x7f978c041bc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 --2- 172.21.15.49:0/661088991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97a01084b0 0x7f97a0197680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 >> 172.21.15.49:0/661088991 conn(0x7f97a0078210 msgr2=0x7f97a01047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:52.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 shutdown_connections 2024-09-17T11:37:52.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:52.624+0000 7f97a7d59640 1 -- 172.21.15.49:0/661088991 wait complete. 2024-09-17T11:37:53.179 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T11:37:53.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T11:37:53.428 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:37:53.543 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:53 smithi049 ceph-mon[30017]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:37:53.543 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:53 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:53.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 -- 172.21.15.49:0/2066775636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4101480 msgr2=0x7feac4101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 --2- 172.21.15.49:0/2066775636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4101480 0x7feac4101860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feab4009880 tx=0x7feab402a5d0 comp rx=0 tx=0).stop 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 -- 172.21.15.49:0/2066775636 shutdown_connections 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 --2- 172.21.15.49:0/2066775636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac4101480 0x7feac4101860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 -- 172.21.15.49:0/2066775636 >> 172.21.15.49:0/2066775636 conn(0x7feac4078280 msgr2=0x7feac40ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 -- 172.21.15.49:0/2066775636 shutdown_connections 2024-09-17T11:37:53.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.880+0000 7feacbd7f640 1 -- 172.21.15.49:0/2066775636 wait complete. 2024-09-17T11:37:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.881+0000 7feacbd7f640 1 Processor -- start 2024-09-17T11:37:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.881+0000 7feacbd7f640 1 -- start start 2024-09-17T11:37:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.881+0000 7feacbd7f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:53.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.881+0000 7feacbd7f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112730 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac9b1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac9b1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46034/0 (socket says 172.21.15.49:46034) 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac9b1b640 1 -- 172.21.15.49:0/167093015 learned_addr learned my addr 172.21.15.49:0/167093015 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac9b1b640 1 -- 172.21.15.49:0/167093015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac419e640 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac9b1b640 1 --2- 172.21.15.49:0/167093015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feab4005ab0 tx=0x7feab40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab403f070 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac419e8d0 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7feab4003a70 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab4035030 con 0x7feac419dd20 2024-09-17T11:37:53.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.882+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41a3850 con 0x7feac419dd20 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.883+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7feab4042070 con 0x7feac419dd20 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.883+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac406ad10 con 0x7feac419dd20 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.883+0000 7feac2ffd640 1 --2- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 0x7feaac041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.883+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (secure 0 0 0) 0x7feab4074ae0 con 0x7feac419dd20 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.884+0000 7feac931a640 1 --2- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 0x7feaac041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:53.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.884+0000 7feac931a640 1 --2- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 0x7feaac041bc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feab80098c0 tx=0x7feab8006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:53.888+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7feab4005590 con 0x7feac419dd20 2024-09-17T11:37:54.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.079+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7feac41018e0 con 0x7feac419dd20 2024-09-17T11:37:54.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.397+0000 7feac2ffd640 1 -- 172.21.15.49:0/167093015 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7feab4036260 con 0x7feac419dd20 2024-09-17T11:37:54.398 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.399+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 msgr2=0x7feaac041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.399+0000 7feacbd7f640 1 --2- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 0x7feaac041bc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feab80098c0 tx=0x7feab8006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.399+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 msgr2=0x7feac419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.399+0000 7feacbd7f640 1 --2- 172.21.15.49:0/167093015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feab4005ab0 tx=0x7feab40048f0 comp rx=0 tx=0).stop 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 shutdown_connections 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 --2- 172.21.15.49:0/167093015 >> v2:172.21.15.49:6800/1112797402 conn(0x7feaac03f700 0x7feaac041bc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 --2- 172.21.15.49:0/167093015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feac419dd20 0x7feac419e100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 >> 172.21.15.49:0/167093015 conn(0x7feac4101480 msgr2=0x7feac40fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 shutdown_connections 2024-09-17T11:37:54.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:37:54.400+0000 7feacbd7f640 1 -- 172.21.15.49:0/167093015 wait complete. 2024-09-17T11:37:54.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:54 smithi049 ceph-mon[30017]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:37:54.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/167093015' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-17T11:37:54.936 INFO:tasks.cephadm:Adding mon.a on smithi049 2024-09-17T11:37:54.936 INFO:tasks.cephadm:Adding mon.c on smithi049 2024-09-17T11:37:54.936 INFO:tasks.cephadm:Adding mon.b on smithi129 2024-09-17T11:37:54.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b' 2024-09-17T11:37:55.257 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-17T11:37:55.303 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/167093015' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"} : dispatch 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:37:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:37:55.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 -- 172.21.15.129:0/2029214806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 msgr2=0x7f77201016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:55.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 --2- 172.21.15.129:0/2029214806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f77201016e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7710009c20 tx=0x7f771002aa90 comp rx=0 tx=0).stop 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 -- 172.21.15.129:0/2029214806 shutdown_connections 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 --2- 172.21.15.129:0/2029214806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f77201016e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 -- 172.21.15.129:0/2029214806 >> 172.21.15.129:0/2029214806 conn(0x7f77200fcbd0 msgr2=0x7f77200feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 -- 172.21.15.129:0/2029214806 shutdown_connections 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.819+0000 7f7724a30640 1 -- 172.21.15.129:0/2029214806 wait complete. 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f7724a30640 1 Processor -- start 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f7724a30640 1 -- start start 2024-09-17T11:37:55.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f7724a30640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f7724a30640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77201069f0 con 0x7f77201012e0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f771e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f771e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.129:41258/0 (socket says 172.21.15.129:41258) 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f771e59c640 1 -- 172.21.15.129:0/1824823340 learned_addr learned my addr 172.21.15.129:0/1824823340 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.820+0000 7f771e59c640 1 -- 172.21.15.129:0/1824823340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772006b430 con 0x7f77201012e0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f771e59c640 1 --2- 172.21.15.129:0/1824823340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7710005e30 tx=0x7f7710030fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7710047070 con 0x7f77201012e0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f7710003df0 con 0x7f77201012e0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772019a030 con 0x7f77201012e0 2024-09-17T11:37:55.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7710033050 con 0x7f77201012e0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.821+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772019a560 con 0x7f77201012e0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.822+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f771004a070 con 0x7f77201012e0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.822+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7720107170 con 0x7f77201012e0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.822+0000 7f77177fe640 1 --2- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 0x7f770c041c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.822+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f771003e030 con 0x7f77201012e0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.823+0000 7f771dd9b640 1 --2- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 0x7f770c041c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.823+0000 7f771dd9b640 1 --2- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 0x7f770c041c10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77080098c0 tx=0x7f7708006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:55.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:55.827+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f7710048260 con 0x7f77201012e0 2024-09-17T11:37:56.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.014+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b", "target": ["mon-mgr", ""]}) -- 0x7f77200ff000 con 0x7f770c03f750 2024-09-17T11:37:56.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.018+0000 7f77177fe640 1 -- 172.21.15.129:0/1824823340 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f77200ff000 con 0x7f770c03f750 2024-09-17T11:37:56.021 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mon update... 2024-09-17T11:37:56.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 msgr2=0x7f770c041c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:56.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 --2- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 0x7f770c041c10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77080098c0 tx=0x7f7708006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:56.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 msgr2=0x7f772006aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:56.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 --2- 172.21.15.129:0/1824823340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7710005e30 tx=0x7f7710030fb0 comp rx=0 tx=0).stop 2024-09-17T11:37:56.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 shutdown_connections 2024-09-17T11:37:56.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 --2- 172.21.15.129:0/1824823340 >> v2:172.21.15.49:6800/1112797402 conn(0x7f770c03f750 0x7f770c041c10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:56.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 --2- 172.21.15.129:0/1824823340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77201012e0 0x7f772006aed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:56.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 >> 172.21.15.129:0/1824823340 conn(0x7f77200fcbd0 msgr2=0x7f77200fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:56.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 shutdown_connections 2024-09-17T11:37:56.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:56.021+0000 7f7724a30640 1 -- 172.21.15.129:0/1824823340 wait complete. 2024-09-17T11:37:56.591 DEBUG:teuthology.orchestra.run.smithi049:mon.c> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T11:37:56.594 DEBUG:teuthology.orchestra.run.smithi129:mon.b> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T11:37:56.596 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T11:37:56.596 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:37:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:37:57.234 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b;count:3 2024-09-17T11:37:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:57 smithi049 ceph-mon[30017]: Deploying daemon mon.b on smithi129 2024-09-17T11:37:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 -- 172.21.15.129:0/3491695784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 msgr2=0x7feca4102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 --2- 172.21.15.129:0/3491695784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4102cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feca0009c20 tx=0x7feca002aa90 comp rx=0 tx=0).stop 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 -- 172.21.15.129:0/3491695784 shutdown_connections 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 --2- 172.21.15.129:0/3491695784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4102cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 -- 172.21.15.129:0/3491695784 >> 172.21.15.129:0/3491695784 conn(0x7feca4078270 msgr2=0x7feca4078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 -- 172.21.15.129:0/3491695784 shutdown_connections 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.992+0000 7fecac573640 1 -- 172.21.15.129:0/3491695784 wait complete. 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.993+0000 7fecac573640 1 Processor -- start 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.993+0000 7fecac573640 1 -- start start 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.993+0000 7fecac573640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.993+0000 7fecac573640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feca410c0e0 con 0x7feca4102910 2024-09-17T11:37:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fecaa30f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fecaa30f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.129:41284/0 (socket says 172.21.15.129:41284) 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fecaa30f640 1 -- 172.21.15.129:0/2835335389 learned_addr learned my addr 172.21.15.129:0/2835335389 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fecaa30f640 1 -- 172.21.15.129:0/2835335389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca4197ba0 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fecaa30f640 1 --2- 172.21.15.129:0/2835335389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feca0005e30 tx=0x7feca002e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.994+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feca0047070 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.995+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7feca0003df0 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.995+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca4197e90 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.995+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feca0033050 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.995+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca419cf40 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.995+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4103a10 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.996+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7feca004a070 con 0x7feca4102910 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.996+0000 7fec9b7fe640 1 --2- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 0x7fec90041c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:37:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.996+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7feca003e030 con 0x7feca4102910 2024-09-17T11:37:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.996+0000 7feca9b0e640 1 --2- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 0x7fec90041c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:37:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:57.996+0000 7feca9b0e640 1 --2- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 0x7fec90041c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fec940098c0 tx=0x7fec94006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:37:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.001+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7feca0048260 con 0x7feca4102910 2024-09-17T11:37:58.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.258+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feca4198790 con 0x7feca4102910 2024-09-17T11:37:58.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.258+0000 7fec9b7fe640 1 -- 172.21.15.129:0/2835335389 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7feca0035d50 con 0x7feca4102910 2024-09-17T11:37:58.260 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:37:58.260 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","modified":"2024-09-17T11:36:40.153935Z","created":"2024-09-17T11:36:40.153935Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T11:37:58.260 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 1 2024-09-17T11:37:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 msgr2=0x7fec90041c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 --2- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 0x7fec90041c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fec940098c0 tx=0x7fec94006db0 comp rx=0 tx=0).stop 2024-09-17T11:37:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 msgr2=0x7feca4197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:37:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 --2- 172.21.15.129:0/2835335389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feca0005e30 tx=0x7feca002e960 comp rx=0 tx=0).stop 2024-09-17T11:37:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 shutdown_connections 2024-09-17T11:37:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 --2- 172.21.15.129:0/2835335389 >> v2:172.21.15.49:6800/1112797402 conn(0x7fec9003f750 0x7fec90041c10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 --2- 172.21.15.129:0/2835335389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feca4102910 0x7feca4197660 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:37:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 >> 172.21.15.129:0/2835335389 conn(0x7feca4078270 msgr2=0x7feca4104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:37:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 shutdown_connections 2024-09-17T11:37:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:37:58.261+0000 7fecac573640 1 -- 172.21.15.129:0/2835335389 wait complete. 2024-09-17T11:37:58.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/2835335389' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:37:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:37:59 smithi049 ceph-mon[30017]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:00.178 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T11:38:00.178 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T11:38:00.511 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:38:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 -- 172.21.15.129:0/1369962225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 msgr2=0x7f9a60101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 --2- 172.21.15.129:0/1369962225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a60101d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009c20 tx=0x7f9a5802aa90 comp rx=0 tx=0).stop 2024-09-17T11:38:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 -- 172.21.15.129:0/1369962225 shutdown_connections 2024-09-17T11:38:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 --2- 172.21.15.129:0/1369962225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a60101d50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 -- 172.21.15.129:0/1369962225 >> 172.21.15.129:0/1369962225 conn(0x7f9a60077170 msgr2=0x7f9a60077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 -- 172.21.15.129:0/1369962225 shutdown_connections 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.943+0000 7f9a64d54640 1 -- 172.21.15.129:0/1369962225 wait complete. 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a64d54640 1 Processor -- start 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a64d54640 1 -- start start 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a64d54640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a64d54640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6010b120 con 0x7f9a60101970 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a5e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a5e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.129:41314/0 (socket says 172.21.15.129:41314) 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.944+0000 7f9a5e59c640 1 -- 172.21.15.129:0/3517842354 learned_addr learned my addr 172.21.15.129:0/3517842354 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a5e59c640 1 -- 172.21.15.129:0/3517842354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6006b5d0 con 0x7f9a60101970 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a5e59c640 1 --2- 172.21.15.129:0/3517842354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009d40 tx=0x7f9a5802e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a58047070 con 0x7f9a60101970 2024-09-17T11:38:01.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f9a58003df0 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a60198b20 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a58033050 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.945+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a60199100 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.946+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60102a90 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.946+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f9a5804a070 con 0x7f9a60101970 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.946+0000 7f9a577fe640 1 --2- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 0x7f9a4c041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:01.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.946+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f9a5803e030 con 0x7f9a60101970 2024-09-17T11:38:01.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.947+0000 7f9a5dd9b640 1 --2- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 0x7f9a4c041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:01.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.947+0000 7f9a5dd9b640 1 --2- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 0x7f9a4c041bc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a480098c0 tx=0x7f9a48006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:01.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:00.952+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9a58004770 con 0x7f9a60101970 2024-09-17T11:38:01.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.207+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9a60101dd0 con 0x7f9a60101970 2024-09-17T11:38:01.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.208+0000 7f9a577fe640 1 -- 172.21.15.129:0/3517842354 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9a58004040 con 0x7f9a60101970 2024-09-17T11:38:01.210 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:38:01.210 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","modified":"2024-09-17T11:36:40.153935Z","created":"2024-09-17T11:36:40.153935Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T11:38:01.210 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 1 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 msgr2=0x7f9a4c041bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 --2- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 0x7f9a4c041bc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a480098c0 tx=0x7f9a48006db0 comp rx=0 tx=0).stop 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 msgr2=0x7f9a601985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 --2- 172.21.15.129:0/3517842354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009d40 tx=0x7f9a5802e960 comp rx=0 tx=0).stop 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 shutdown_connections 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 --2- 172.21.15.129:0/3517842354 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9a4c03f700 0x7f9a4c041bc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 --2- 172.21.15.129:0/3517842354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a60101970 0x7f9a601985e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 >> 172.21.15.129:0/3517842354 conn(0x7f9a60077170 msgr2=0x7f9a601078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 shutdown_connections 2024-09-17T11:38:01.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:01.210+0000 7f9a64d54640 1 -- 172.21.15.129:0/3517842354 wait complete. 2024-09-17T11:38:01.391 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 podman[33313]: 2024-09-17 11:38:01.277927771 +0000 UTC m=+0.116389346 container create 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-17T11:38:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:01 smithi049 ceph-mon[30017]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/3517842354' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 podman[33313]: 2024-09-17 11:38:01.480013406 +0000 UTC m=+0.318474946 container init 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 podman[33313]: 2024-09-17 11:38:01.484796693 +0000 UTC m=+0.323258227 container start 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=main-7498493, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev), process ceph-mon, pid 7 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: pidfile_write: ignore empty --pid-file 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: load: jerasure load: lrc 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: RocksDB version: 7.9.2 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Git sha 0 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Compile date 2024-09-15 20:52:36 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: DB SUMMARY 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: DB Session ID: V4B4DS5MABWLRQDSYSD5 2024-09-17T11:38:01.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: CURRENT file: CURRENT 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.error_if_exists: 0 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.create_if_missing: 0 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.paranoid_checks: 1 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.env: 0x563fbc583120 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T11:38:01.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.info_log: 0x563fbe3d3700 2024-09-17T11:38:01.912 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T11:38:01.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.statistics: (nil) 2024-09-17T11:38:01.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.use_fsync: 0 2024-09-17T11:38:01.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_log_file_size: 0 2024-09-17T11:38:01.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T11:38:01.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_fallocate: 1 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.use_direct_reads: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.db_log_dir: 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.wal_dir: 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T11:38:01.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.write_buffer_manager: 0x563fbe3d7b80 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T11:38:01.915 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.unordered_write: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.row_cache: None 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.wal_filter: None 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.two_write_queues: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.wal_compression: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.atomic_flush: 0 2024-09-17T11:38:01.916 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.log_readahead_size: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_background_jobs: 2 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_background_compactions: -1 2024-09-17T11:38:01.917 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_subcompactions: 1 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_open_files: -1 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_background_flushes: -1 2024-09-17T11:38:01.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Compression algorithms supported: 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kZSTD supported: 0 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kXpressCompression supported: 0 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kBZip2Compression supported: 0 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kLZ4Compression supported: 1 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kZlibCompression supported: 1 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: kSnappyCompression supported: 1 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T11:38:01.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.merge_operator: 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_filter: None 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563fbe3d3320) 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_type: 0 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_index_type: 0 2024-09-17T11:38:01.920 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_shortening: 1 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: checksum: 4 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: no_block_cache: 0 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache: 0x563fbe3c7610 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_options: 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: capacity : 536870912 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_shard_bits : 4 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: strict_capacity_limit : 0 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_compressed: (nil) 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: persistent_cache: (nil) 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size: 4096 2024-09-17T11:38:01.921 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size_deviation: 10 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_restart_interval: 16 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_block_restart_interval: 1 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: metadata_block_size: 4096 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: partition_filters: 0 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: use_delta_encoding: 1 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: filter_policy: bloomfilter 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: whole_key_filtering: 1 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: verify_compression: 0 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: format_version: 5 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_index_compression: 1 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_align: 0 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: max_auto_readahead_size: 262144 2024-09-17T11:38:01.922 INFO:journalctl@ceph.mon.b.smithi129.stdout: prepopulate_block_cache: 0 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout: initial_auto_readahead_size: 8192 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression: NoCompression 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.num_levels: 7 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T11:38:01.923 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:38:01.924 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T11:38:01.925 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T11:38:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T11:38:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.inplace_update_support: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.bloom_locality: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.max_successive_merges: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.ttl: 2592000 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T11:38:01.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enable_blob_files: false 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.min_blob_size: 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d3f449e-378c-4b0a-a45a-8b0533412a5b 2024-09-17T11:38:01.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573081511678, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573081512103, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726573081, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d3f449e-378c-4b0a-a45a-8b0533412a5b", "db_session_id": "V4B4DS5MABWLRQDSYSD5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573081512196, "job": 1, "event": "recovery_finished"} 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563fbe3f6e00 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: DB pointer 0x563fbe4f8000 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** DB Stats ** 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:01.930 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-17T11:38:01.931 INFO:journalctl@ceph.mon.b.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:38:01.932 INFO:journalctl@ceph.mon.b.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache BinnedLRUCache@0x563fbe3c7610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: using public_addr v2:172.21.15.129:0/0 -> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: starting mon.b rank -1 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(???) e0 preinit fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: e1 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: btime 2024-09-17T11:36:43:058780+0000 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T11:38:01.933 INFO:journalctl@ceph.mon.b.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout: legacy client fscid: -1 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout: No filesystems configured 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:01.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/167093015' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"} : dispatch 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.935 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b;count:3 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: Deploying daemon mon.b on smithi129 2024-09-17T11:38:01.936 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/2835335389' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/3517842354' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 ceph-mon[33364]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 bash[33313]: 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 2024-09-17T11:38:01.937 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:01 smithi129 systemd[1]: Started Ceph mon.b for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:38:03.296 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T11:38:03.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T11:38:03.560 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:38:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: mon.a calling monitor election 2024-09-17T11:38:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: mon.b calling monitor election 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T11:38:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: monmap epoch 2 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: last_changed 2024-09-17T11:38:01.536310+0000 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: min_mon_release 19 (squid) 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: election_strategy: 1 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: fsmap 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: mgrmap e13: y(active, since 34s) 2024-09-17T11:38:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: overall HEALTH_OK 2024-09-17T11:38:06.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:06.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: mon.a calling monitor election 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: mon.b calling monitor election 2024-09-17T11:38:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: monmap epoch 2 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:06.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: last_changed 2024-09-17T11:38:01.536310+0000 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: min_mon_release 19 (squid) 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: election_strategy: 1 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: fsmap 2024-09-17T11:38:06.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: mgrmap e13: y(active, since 34s) 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: overall HEALTH_OK 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:06.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 -- 172.21.15.129:0/442475255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 msgr2=0x7f00440059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 --2- 172.21.15.129:0/442475255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00440059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f00580784b0 tx=0x7f004802bc80 comp rx=0 tx=0).stop 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 -- 172.21.15.129:0/442475255 shutdown_connections 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 --2- 172.21.15.129:0/442475255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00440059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 -- 172.21.15.129:0/442475255 >> 172.21.15.129:0/442475255 conn(0x7f0058077170 msgr2=0x7f0058077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 -- 172.21.15.129:0/442475255 shutdown_connections 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.995+0000 7f005d850640 1 -- 172.21.15.129:0/442475255 wait complete. 2024-09-17T11:38:06.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.996+0000 7f005d850640 1 Processor -- start 2024-09-17T11:38:06.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.996+0000 7f005d850640 1 -- start start 2024-09-17T11:38:06.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.996+0000 7f005d850640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:06.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.996+0000 7f005d850640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:06.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f005d850640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0058109cb0 con 0x7f0044003530 2024-09-17T11:38:06.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f005d850640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0058109b30 con 0x7f0058197d30 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f00567fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.129:35590/0 (socket says 172.21.15.129:35590) 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 -- 172.21.15.129:0/933489635 learned_addr learned my addr 172.21.15.129:0/933489635 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f00567fc640 1 -- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 msgr2=0x7f005819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f00567fc640 1 -- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 msgr2=0x7f005819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f00567fc640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f00567fc640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 -- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 msgr2=0x7f005819d0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:38:07.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 -- 172.21.15.129:0/933489635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00581983b0 con 0x7f0044003530 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f0056ffd640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00480092e0 tx=0x7f0048004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.997+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f004802d6a0 con 0x7f0044003530 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.998+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f004802d840 con 0x7f0044003530 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.998+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0048005470 con 0x7f0044003530 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.998+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005819d670 con 0x7f0044003530 2024-09-17T11:38:07.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.998+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005819dc30 con 0x7f0044003530 2024-09-17T11:38:07.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.999+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f0048036070 con 0x7f0044003530 2024-09-17T11:38:07.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.999+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005819dee0 con 0x7f0044003530 2024-09-17T11:38:07.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.999+0000 7f005c84e640 1 --2- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 0x7f00380419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:07.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.999+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f0048075690 con 0x7f0044003530 2024-09-17T11:38:07.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:06.999+0000 7f00567fc640 1 --2- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 0x7f00380419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:07.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.000+0000 7f00567fc640 1 --2- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 0x7f00380419b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0040006fd0 tx=0x7f0040008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:07.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.004+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f004802c340 con 0x7f0044003530 2024-09-17T11:38:07.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.249+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0058191300 con 0x7f0044003530 2024-09-17T11:38:07.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.250+0000 7f005c84e640 1 -- 172.21.15.129:0/933489635 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f004803fdb0 con 0x7f0044003530 2024-09-17T11:38:07.251 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:38:07.252 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","modified":"2024-09-17T11:38:01.536310Z","created":"2024-09-17T11:36:40.153935Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T11:38:07.252 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 2 2024-09-17T11:38:07.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 msgr2=0x7f00380419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:07.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 --2- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 0x7f00380419b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0040006fd0 tx=0x7f0040008040 comp rx=0 tx=0).stop 2024-09-17T11:38:07.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 msgr2=0x7f00581977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:07.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00480092e0 tx=0x7f0048004c10 comp rx=0 tx=0).stop 2024-09-17T11:38:07.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 shutdown_connections 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 --2- 172.21.15.129:0/933489635 >> v2:172.21.15.49:6800/1112797402 conn(0x7f003803f4f0 0x7f00380419b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0058197d30 0x7f005819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 --2- 172.21.15.129:0/933489635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0044003530 0x7f00581977f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 >> 172.21.15.129:0/933489635 conn(0x7f0058077170 msgr2=0x7f00581089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 shutdown_connections 2024-09-17T11:38:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:07.252+0000 7f005d850640 1 -- 172.21.15.129:0/933489635 wait complete. 2024-09-17T11:38:07.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:07 smithi129 ceph-mon[33364]: Deploying daemon mon.c on smithi049 2024-09-17T11:38:07.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/933489635' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:07.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:07 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:07 smithi049 ceph-mon[30017]: Deploying daemon mon.c on smithi049 2024-09-17T11:38:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/933489635' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:07 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:08.844 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T11:38:08.844 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T11:38:08.863 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:08 smithi049 ceph-mon[30017]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:08.872 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:08 smithi129 ceph-mon[33364]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:09.096 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:38:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 systemd[1]: Starting Ceph mon.c for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.489+0000 7f927315f640 1 -- 172.21.15.129:0/29028327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c101970 msgr2=0x7f926c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.489+0000 7f927315f640 1 --2- 172.21.15.129:0/29028327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c101970 0x7f926c101d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9268009880 tx=0x7f926802a5d0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.489+0000 7f927315f640 1 -- 172.21.15.129:0/29028327 shutdown_connections 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.489+0000 7f927315f640 1 --2- 172.21.15.129:0/29028327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c101970 0x7f926c101d50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.489+0000 7f927315f640 1 -- 172.21.15.129:0/29028327 >> 172.21.15.129:0/29028327 conn(0x7f926c077170 msgr2=0x7f926c077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 -- 172.21.15.129:0/29028327 shutdown_connections 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 -- 172.21.15.129:0/29028327 wait complete. 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 Processor -- start 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 -- start start 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:09.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10b120 con 0x7f926c196d40 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.490+0000 7f927315f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10afa0 con 0x7f926c101970 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.129:35602/0 (socket says 172.21.15.129:35602) 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 -- 172.21.15.129:0/1711852147 learned_addr learned my addr 172.21.15.129:0/1711852147 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9270efb640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9270efb640 1 -- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 msgr2=0x7f926c196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9270efb640 1 -- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 msgr2=0x7f926c196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9270efb640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9270efb640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 -- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 msgr2=0x7f926c196800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 -- 172.21.15.129:0/1711852147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1973c0 con 0x7f926c196d40 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9263fff640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b4a0 tx=0x7f925c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:09.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f925c010610 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f925c010c30 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c19c680 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.491+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f925c017c40 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.492+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c19cc40 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.492+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f925c017070 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.492+0000 7f9261ffb640 1 --2- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 0x7f9254043bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.493+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f925c00d040 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.493+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c102a90 con 0x7f926c196d40 2024-09-17T11:38:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.493+0000 7f9270efb640 1 --2- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 0x7f9254043bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:09.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.493+0000 7f9270efb640 1 --2- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 0x7f9254043bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92680096c0 tx=0x7f92680058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:09.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.498+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f925c01e030 con 0x7f926c196d40 2024-09-17T11:38:09.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.739+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f926c19cf50 con 0x7f926c196d40 2024-09-17T11:38:09.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.740+0000 7f9261ffb640 1 -- 172.21.15.129:0/1711852147 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f925c0107b0 con 0x7f926c196d40 2024-09-17T11:38:09.741 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:38:09.741 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","modified":"2024-09-17T11:38:01.536310Z","created":"2024-09-17T11:36:40.153935Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T11:38:09.741 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 2 2024-09-17T11:38:09.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 msgr2=0x7f9254043bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 --2- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 0x7f9254043bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92680096c0 tx=0x7f92680058b0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 msgr2=0x7f926c19c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b4a0 tx=0x7f925c00b960 comp rx=0 tx=0).stop 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 shutdown_connections 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 --2- 172.21.15.129:0/1711852147 >> v2:172.21.15.49:6800/1112797402 conn(0x7f9254041710 0x7f9254043bd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f926c196d40 0x7f926c19c0e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 --2- 172.21.15.129:0/1711852147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f926c101970 0x7f926c196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 >> 172.21.15.129:0/1711852147 conn(0x7f926c077170 msgr2=0x7f926c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 shutdown_connections 2024-09-17T11:38:09.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:09.742+0000 7f927315f640 1 -- 172.21.15.129:0/1711852147 wait complete. 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 podman[40185]: 2024-09-17 11:38:09.497959448 +0000 UTC m=+0.101770322 container create 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c, RELEASE=main-7498493, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 podman[40185]: 2024-09-17 11:38:09.412754059 +0000 UTC m=+0.016564935 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 podman[40185]: 2024-09-17 11:38:09.717188254 +0000 UTC m=+0.320999138 container init 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 podman[40185]: 2024-09-17 11:38:09.722187741 +0000 UTC m=+0.325998617 container start 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-7498493, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: ceph version 19.3.0-4937-g74984932 (7498493279982dfe87f00616198a5967475f6169) squid (dev), process ceph-mon, pid 7 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pidfile_write: ignore empty --pid-file 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: load: jerasure load: lrc 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: RocksDB version: 7.9.2 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Git sha 0 2024-09-17T11:38:09.750 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Compile date 2024-09-15 20:52:36 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: DB SUMMARY 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: DB Session ID: O54TNZMZV3DNVJIGDMIF 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: CURRENT file: CURRENT 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.error_if_exists: 0 2024-09-17T11:38:09.751 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.create_if_missing: 0 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.paranoid_checks: 1 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.env: 0x56120ac7d120 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.info_log: 0x56120c49f700 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.statistics: (nil) 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.use_fsync: 0 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_log_file_size: 0 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T11:38:09.752 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_fallocate: 1 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.use_direct_reads: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.db_log_dir: 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.wal_dir: 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T11:38:09.753 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.write_buffer_manager: 0x56120c4a3b80 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.unordered_write: 0 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T11:38:09.754 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.row_cache: None 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.wal_filter: None 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.two_write_queues: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.wal_compression: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.atomic_flush: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.log_readahead_size: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T11:38:09.755 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_background_jobs: 2 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_background_compactions: -1 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_subcompactions: 1 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T11:38:09.756 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_open_files: -1 2024-09-17T11:38:09.757 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T11:38:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T11:38:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T11:38:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_background_flushes: -1 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Compression algorithms supported: 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kZSTD supported: 0 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kXpressCompression supported: 0 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kBZip2Compression supported: 0 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kLZ4Compression supported: 1 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kZlibCompression supported: 1 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: kSnappyCompression supported: 1 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T11:38:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.merge_operator: 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_filter: None 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56120c49f320) 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T11:38:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: index_type: 0 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: data_block_index_type: 0 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: index_shortening: 1 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: checksum: 4 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: no_block_cache: 0 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_cache: 0x56120c493610 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_cache_options: 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: capacity : 536870912 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: num_shard_bits : 4 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: strict_capacity_limit : 0 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-09-17T11:38:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_cache_compressed: (nil) 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: persistent_cache: (nil) 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_size: 4096 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_size_deviation: 10 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_restart_interval: 16 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: index_block_restart_interval: 1 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: metadata_block_size: 4096 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: partition_filters: 0 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: use_delta_encoding: 1 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: filter_policy: bloomfilter 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: whole_key_filtering: 1 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: verify_compression: 0 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: format_version: 5 2024-09-17T11:38:10.154 INFO:journalctl@ceph.mon.c.smithi049.stdout: enable_index_compression: 1 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout: block_align: 0 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout: max_auto_readahead_size: 262144 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout: prepopulate_block_cache: 0 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout: initial_auto_readahead_size: 8192 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression: NoCompression 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.num_levels: 7 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T11:38:10.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T11:38:10.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T11:38:10.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T11:38:10.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T11:38:10.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T11:38:10.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T11:38:10.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T11:38:10.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T11:38:10.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T11:38:10.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.inplace_update_support: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.bloom_locality: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.max_successive_merges: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.ttl: 2592000 2024-09-17T11:38:10.161 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enable_blob_files: false 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.min_blob_size: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T11:38:10.162 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2cde1bc0-60d0-4b0f-8cc1-c5887ffe16ae 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573089750196, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573089750558, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726573089, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2cde1bc0-60d0-4b0f-8cc1-c5887ffe16ae", "db_session_id": "O54TNZMZV3DNVJIGDMIF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726573089750658, "job": 1, "event": "recovery_finished"} 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56120c4c2e00 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: DB pointer 0x56120c5c4000 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T11:38:10.163 INFO:journalctl@ceph.mon.c.smithi049.stdout: ** DB Stats ** 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: ** Compaction Stats [default] ** 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:10.164 INFO:journalctl@ceph.mon.c.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: ** Compaction Stats [default] ** 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T11:38:10.165 INFO:journalctl@ceph.mon.c.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: Block cache BinnedLRUCache@0x56120c493610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: using public_addrv [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: starting mon.c rank -1 at public addrs [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] at bind addrs [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(???) e0 preinit fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).mds e1 new map 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-17T11:38:10.166 INFO:journalctl@ceph.mon.c.smithi049.stdout: e1 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: btime 2024-09-17T11:36:43:058780+0000 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: legacy client fscid: -1 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout: No filesystems configured 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T11:38:10.167 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/167093015' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"} : dispatch 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:10.168 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:10.169 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi049:[v2:172.21.15.49:3301,v1:172.21.15.49:6790]=c;smithi129:172.21.15.129=b;count:3 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Deploying daemon mon.b on smithi129 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/2835335389' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/3517842354' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:10.170 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.a calling monitor election 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.b calling monitor election 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T11:38:10.171 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: monmap epoch 2 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: last_changed 2024-09-17T11:38:01.536310+0000 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: min_mon_release 19 (squid) 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: election_strategy: 1 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: fsmap 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mgrmap e13: y(active, since 34s) 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: overall HEALTH_OK 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.172 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: Deploying daemon mon.c on smithi049 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/933489635' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 ceph-mon[40223]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 bash[40185]: 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 2024-09-17T11:38:10.173 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:09 smithi049 systemd[1]: Started Ceph mon.c for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:38:11.338 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T11:38:11.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T11:38:11.592 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:38:14.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.981+0000 7fbeb7304640 1 -- 172.21.15.129:0/2386038064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0101970 msgr2=0x7fbe9c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.981+0000 7fbeb7304640 1 --2- 172.21.15.129:0/2386038064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0101970 0x7fbe9c005580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbea0011c50 tx=0x7fbea002be50 comp rx=0 tx=0).stop 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.981+0000 7fbeb7304640 1 -- 172.21.15.129:0/2386038064 shutdown_connections 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.981+0000 7fbeb7304640 1 --2- 172.21.15.129:0/2386038064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb0101970 0x7fbe9c005580 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.981+0000 7fbeb7304640 1 -- 172.21.15.129:0/2386038064 >> 172.21.15.129:0/2386038064 conn(0x7fbeb0077170 msgr2=0x7fbeb0077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.982+0000 7fbeb7304640 1 -- 172.21.15.129:0/2386038064 shutdown_connections 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.982+0000 7fbeb7304640 1 -- 172.21.15.129:0/2386038064 wait complete. 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.982+0000 7fbeb7304640 1 Processor -- start 2024-09-17T11:38:14.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.982+0000 7fbeb7304640 1 -- start start 2024-09-17T11:38:14.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 0x7fbeb01a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0108bd0 con 0x7fbeb01a3a20 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0108a50 con 0x7fbeb00fed50 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb7304640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb0108d50 con 0x7fbeb0101970 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41074/0 (socket says 172.21.15.129:41074) 2024-09-17T11:38:14.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 -- 172.21.15.129:0/3879804526 learned_addr learned my addr 172.21.15.129:0/3879804526 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 0x7fbeb01a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb50a0640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 msgr2=0x7fbeb00ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 msgr2=0x7fbeb00ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb489f640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb50a0640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 msgr2=0x7fbeb00fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb50a0640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 msgr2=0x7fbeb00fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb50a0640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb50a0640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 msgr2=0x7fbeb00fe810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 msgr2=0x7fbeb00ff190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 -- 172.21.15.129:0/3879804526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01a6450 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.983+0000 7fbeb58a1640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 0x7fbeb01a5e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bd50 tx=0x7fbea800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.984+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8019070 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.984+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fbea80148c0 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.984+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80127b0 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.984+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb01a6740 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.984+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01a6cb0 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.985+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7fbea801c070 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.985+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb0109b90 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.985+0000 7fbea67fc640 1 --2- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 0x7fbe94042050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.985+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7fbea8053b60 con 0x7fbeb01a3a20 2024-09-17T11:38:14.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.985+0000 7fbeb50a0640 1 --2- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 0x7fbe94042050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:14.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.986+0000 7fbeb50a0640 1 --2- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 0x7fbe94042050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0010c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:14.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:14.990+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbea80133a0 con 0x7fbeb01a3a20 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: mon.a calling monitor election 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: mon.b calling monitor election 2024-09-17T11:38:15.130 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: mon.c calling monitor election 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: monmap epoch 3 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: last_changed 2024-09-17T11:38:09.776037+0000 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: min_mon_release 19 (squid) 2024-09-17T11:38:15.131 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: election_strategy: 1 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: 2: [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon.c 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: fsmap 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: mgrmap e13: y(active, since 43s) 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: overall HEALTH_OK 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:15.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: mon.a calling monitor election 2024-09-17T11:38:15.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:15.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: mon.b calling monitor election 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: mon.c calling monitor election 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: monmap epoch 3 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: last_changed 2024-09-17T11:38:09.776037+0000 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: min_mon_release 19 (squid) 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: election_strategy: 1 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: 2: [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon.c 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: fsmap 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: mgrmap e13: y(active, since 43s) 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: overall HEALTH_OK 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: mon.a calling monitor election 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:38:15.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: mon.b calling monitor election 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: mon.c calling monitor election 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: monmap epoch 3 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: fsid f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:15.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: last_changed 2024-09-17T11:38:09.776037+0000 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: created 2024-09-17T11:36:40.153935+0000 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: min_mon_release 19 (squid) 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: election_strategy: 1 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: 2: [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon.c 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: fsmap 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: mgrmap e13: y(active, since 43s) 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: overall HEALTH_OK 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:15.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.270+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbeb01a7000 con 0x7fbeb01a3a20 2024-09-17T11:38:15.273 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:38:15.273 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","modified":"2024-09-17T11:38:09.776037Z","created":"2024-09-17T11:36:40.153935Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3301","nonce":0},{"type":"v1","addr":"172.21.15.49:6790","nonce":0}]},"addr":"172.21.15.49:6790/0","public_addr":"172.21.15.49:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-17T11:38:15.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.271+0000 7fbea67fc640 1 -- 172.21.15.129:0/3879804526 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fbea800e0a0 con 0x7fbeb01a3a20 2024-09-17T11:38:15.273 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 3 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 msgr2=0x7fbe94042050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 0x7fbe94042050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0010c50 comp rx=0 tx=0).stop 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 msgr2=0x7fbeb01a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 0x7fbeb01a5e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bd50 tx=0x7fbea800be40 comp rx=0 tx=0).stop 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 shutdown_connections 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe9403fb90 0x7fbe94042050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbeb01a3a20 0x7fbeb01a5e10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:15.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb00fed50 0x7fbeb00ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:15.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 --2- 172.21.15.129:0/3879804526 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbeb0101970 0x7fbeb00fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:15.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.273+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 >> 172.21.15.129:0/3879804526 conn(0x7fbeb0077170 msgr2=0x7fbeb01058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:15.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.274+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 shutdown_connections 2024-09-17T11:38:15.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:15.274+0000 7fbeb7304640 1 -- 172.21.15.129:0/3879804526 wait complete. 2024-09-17T11:38:15.969 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:15.969 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/3879804526' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:15.969 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:15.970 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T11:38:15.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/3879804526' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/3879804526' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-17T11:38:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:38:16.229 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 -- 172.21.15.49:0/147360213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c102780 msgr2=0x7f3e0c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/147360213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c102780 0x7f3e0c102b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009880 tx=0x7f3e0002a5d0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 -- 172.21.15.49:0/147360213 shutdown_connections 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/147360213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c102780 0x7f3e0c102b60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 -- 172.21.15.49:0/147360213 >> 172.21.15.49:0/147360213 conn(0x7f3e0c0fe1d0 msgr2=0x7f3e0c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 -- 172.21.15.49:0/147360213 shutdown_connections 2024-09-17T11:38:16.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.668+0000 7f3e12ff1640 1 -- 172.21.15.49:0/147360213 wait complete. 2024-09-17T11:38:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.669+0000 7f3e12ff1640 1 Processor -- start 2024-09-17T11:38:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.669+0000 7f3e12ff1640 1 -- start start 2024-09-17T11:38:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 0x7f3e0c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37300/0 (socket says 172.21.15.49:37300) 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 0x7f3e0c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e0bfff640 1 -- 172.21.15.49:0/1383235487 learned_addr learned my addr 172.21.15.49:0/1383235487 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c10c0e0 con 0x7f3e0c197d80 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e10d8d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 0x7f3e0c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c10bf60 con 0x7f3e0c102780 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0c10c260 con 0x7f3e0c19d660 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e1158e640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 0x7f3e0c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.670+0000 7f3e10d8d640 1 -- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 msgr2=0x7f3e0c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e10d8d640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 0x7f3e0c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e10d8d640 1 -- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 msgr2=0x7f3e0c19d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e10d8d640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e10d8d640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c1a42a0 con 0x7f3e0c102780 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e0bfff640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e1158e640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 0x7f3e0c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e10d8d640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 0x7f3e0c197840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006010 tx=0x7f3e00005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0003f070 con 0x7f3e0c102780 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3e00005630 con 0x7f3e0c102780 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c1a4590 con 0x7f3e0c102780 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.671+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00035030 con 0x7f3e0c102780 2024-09-17T11:38:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.672+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c1a4b20 con 0x7f3e0c102780 2024-09-17T11:38:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.673+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c102b60 con 0x7f3e0c102780 2024-09-17T11:38:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.673+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f3e00042070 con 0x7f3e0c102780 2024-09-17T11:38:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.673+0000 7f3e09ffb640 1 --2- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 0x7f3dfc042050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.673+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f3e00074c50 con 0x7f3e0c102780 2024-09-17T11:38:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.673+0000 7f3e0bfff640 1 --2- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 0x7f3dfc042050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.674+0000 7f3e0bfff640 1 --2- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 0x7f3dfc042050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009f70 tx=0x7f3df8009240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.678+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3e000319f0 con 0x7f3e0c102780 2024-09-17T11:38:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.884+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3e0c10a930 con 0x7f3e0c102780 2024-09-17T11:38:16.886 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:38:16.775+0000 7f3f808d0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T11:38:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.884+0000 7f3e09ffb640 1 -- 172.21.15.49:0/1383235487 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f3e0004c350 con 0x7f3e0c102780 2024-09-17T11:38:16.887 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:16.887 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-09-17T11:38:16.887 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:38:16.887 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 msgr2=0x7f3dfc042050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 0x7f3dfc042050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009f70 tx=0x7f3df8009240 comp rx=0 tx=0).stop 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 msgr2=0x7f3e0c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 0x7f3e0c197840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e00006010 tx=0x7f3e00005230 comp rx=0 tx=0).stop 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 shutdown_connections 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3dfc03fb90 0x7f3dfc042050 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3e0c19d660 0x7f3e0c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e0c197d80 0x7f3e0c19d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 --2- 172.21.15.49:0/1383235487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e0c102780 0x7f3e0c197840 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:16.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 >> 172.21.15.49:0/1383235487 conn(0x7f3e0c0fe1d0 msgr2=0x7f3e0c104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:16.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 shutdown_connections 2024-09-17T11:38:16.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:16.887+0000 7f3e12ff1640 1 -- 172.21.15.49:0/1383235487 wait complete. 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:17.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1383235487' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1383235487' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:17.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1383235487' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:17.533 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T11:38:17.533 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:38:17.533 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T11:38:17.561 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:38:17.561 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:38:17.628 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:38:17.628 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T11:38:17.661 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:38:17.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:38:17.731 INFO:tasks.cephadm:Adding mgr.y on smithi049 2024-09-17T11:38:17.731 INFO:tasks.cephadm:Adding mgr.x on smithi129 2024-09-17T11:38:17.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi049=y;smithi129=x' 2024-09-17T11:38:18.018 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[40223]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[40223]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[40223]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[40223]: Reconfiguring daemon mon.a on smithi049 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:18.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[30017]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[30017]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[30017]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T11:38:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[30017]: Reconfiguring daemon mon.a on smithi049 2024-09-17T11:38:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:18 smithi129 ceph-mon[33364]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:18 smithi129 ceph-mon[33364]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:38:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:18 smithi129 ceph-mon[33364]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T11:38:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:18 smithi129 ceph-mon[33364]: Reconfiguring daemon mon.a on smithi049 2024-09-17T11:38:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 -- 172.21.15.129:0/3806106556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc006b350 msgr2=0x7f5fc0105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3806106556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc006b350 0x7f5fc0105130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009c20 tx=0x7f5fb002aa90 comp rx=0 tx=0).stop 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 -- 172.21.15.129:0/3806106556 shutdown_connections 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3806106556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc006b350 0x7f5fc0105130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 -- 172.21.15.129:0/3806106556 >> 172.21.15.129:0/3806106556 conn(0x7f5fc00fcd20 msgr2=0x7f5fc00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 -- 172.21.15.129:0/3806106556 shutdown_connections 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 -- 172.21.15.129:0/3806106556 wait complete. 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.436+0000 7f5fc5589640 1 Processor -- start 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 -- start start 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc0197ae0 0x7f5fc01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 0x7f5fc01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0107c50 con 0x7f5fc0197ae0 2024-09-17T11:38:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0107ad0 con 0x7f5fc006b350 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fc5589640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc0107dd0 con 0x7f5fc0198d00 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41086/0 (socket says 172.21.15.129:41086) 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 -- 172.21.15.129:0/3743619979 learned_addr learned my addr 172.21.15.129:0/3743619979 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbe7fc640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc0197ae0 0x7f5fc01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbf7fe640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 0x7f5fc01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 -- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 msgr2=0x7f5fc01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 0x7f5fc01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 -- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc0197ae0 msgr2=0x7f5fc01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc0197ae0 0x7f5fc01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.437+0000 7f5fbeffd640 1 -- 172.21.15.129:0/3743619979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc019f8e0 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fbf7fe640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 0x7f5fc01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fbeffd640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002ea70 tx=0x7f5fb002f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0047070 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f5fb002e730 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0033050 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc019fb70 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.438+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc01a00a0 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.439+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7f5fb004a070 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.439+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc01031a0 con 0x7f5fc006b350 2024-09-17T11:38:18.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.439+0000 7f5fb7fff640 1 --2- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 0x7f5fac042050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:18.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.439+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7f5fb0075040 con 0x7f5fc006b350 2024-09-17T11:38:18.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.439+0000 7f5fbe7fc640 1 --2- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 0x7f5fac042050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:18.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.440+0000 7f5fbe7fc640 1 --2- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 0x7f5fac042050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009a30 tx=0x7f5fa8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:18.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.444+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f5fb00373b0 con 0x7f5fc006b350 2024-09-17T11:38:18.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.637+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi049=y;smithi129=x", "target": ["mon-mgr", ""]}) -- 0x7f5fc0063590 con 0x7f5fac03fb90 2024-09-17T11:38:18.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.643+0000 7f5fb7fff640 1 -- 172.21.15.129:0/3743619979 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5fc0063590 con 0x7f5fac03fb90 2024-09-17T11:38:18.645 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mgr update... 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 msgr2=0x7f5fac042050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 0x7f5fac042050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009a30 tx=0x7f5fa8009190 comp rx=0 tx=0).stop 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 msgr2=0x7f5fc0197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002ea70 tx=0x7f5fb002f8a0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 shutdown_connections 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> v2:172.21.15.49:6800/1112797402 conn(0x7f5fac03fb90 0x7f5fac042050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5fc0198d00 0x7f5fc01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc0197ae0 0x7f5fc01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 --2- 172.21.15.129:0/3743619979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fc006b350 0x7f5fc0197530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:18.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.645+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 >> 172.21.15.129:0/3743619979 conn(0x7f5fc00fcd20 msgr2=0x7f5fc00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:18.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.646+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 shutdown_connections 2024-09-17T11:38:18.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:38:18.646+0000 7f5fc5589640 1 -- 172.21.15.129:0/3743619979 wait complete. 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:19.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:19 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.244 DEBUG:teuthology.orchestra.run.smithi129:mgr.x> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T11:38:19.247 DEBUG:tasks.cephadm:set 0 configs 2024-09-17T11:38:19.247 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T11:38:19.247 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:38:19.247 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T11:38:19.263 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T11:38:19.264 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-09-17T11:38:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T11:38:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:19.319 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-17T11:38:19.319 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:19.320 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:19.320 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-17 11:38:09.235925873 +0000 2024-09-17T11:38:19.320 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-17 11:33:23.080800518 +0000 2024-09-17T11:38:19.320 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-17 11:33:23.080800518 +0000 2024-09-17T11:38:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-17 11:33:23.080800518 +0000 2024-09-17T11:38:19.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T11:38:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:19.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:19.382 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:19 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:19.384 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-17T11:38:19.385 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-17T11:38:19.385 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.575e-05 s, 5.3 MB/s 2024-09-17T11:38:19.386 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T11:38:19.443 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-09-17T11:38:19.499 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-17 11:38:09.236925846 +0000 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-17 11:33:23.077800918 +0000 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-17 11:33:23.077800918 +0000 2024-09-17T11:38:19.500 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-17 11:33:23.077800918 +0000 2024-09-17T11:38:19.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T11:38:19.564 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-17T11:38:19.564 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-17T11:38:19.564 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.3856e-05 s, 5.5 MB/s 2024-09-17T11:38:19.566 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T11:38:19.622 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-09-17T11:38:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T11:38:19.678 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-17 11:38:09.236925846 +0000 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-17 11:33:23.082800252 +0000 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-17 11:33:23.082800252 +0000 2024-09-17T11:38:19.679 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-17 11:33:23.082800252 +0000 2024-09-17T11:38:19.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T11:38:19.742 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-17T11:38:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-17T11:38:19.743 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.171e-05 s, 5.6 MB/s 2024-09-17T11:38:19.744 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T11:38:19.801 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-09-17 11:38:09.236925846 +0000 2024-09-17T11:38:19.858 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-09-17 11:33:23.079800652 +0000 2024-09-17T11:38:19.860 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-09-17 11:33:23.079800652 +0000 2024-09-17T11:38:19.860 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-09-17 11:33:23.079800652 +0000 2024-09-17T11:38:19.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T11:38:19.922 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-09-17T11:38:19.922 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-09-17T11:38:19.923 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000110064 s, 4.7 MB/s 2024-09-17T11:38:19.924 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T11:38:19.980 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:38:19.980 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T11:38:19.996 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T11:38:19.997 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-17T11:38:20.053 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 11:38:00.416806073 +0000 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.054 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.054 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T11:38:20.119 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T11:38:20.119 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T11:38:20.119 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000257627 s, 2.0 MB/s 2024-09-17T11:38:20.121 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T11:38:20.177 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-17T11:38:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T11:38:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 11:38:00.417806051 +0000 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T11:38:20.297 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T11:38:20.298 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T11:38:20.298 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000188162 s, 2.7 MB/s 2024-09-17T11:38:20.299 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T11:38:20.355 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-17T11:38:20.378 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: Reconfiguring mon.c (monmap changed)... 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: Reconfiguring daemon mon.c on smithi049 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi049=y;smithi129=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: Saving service mgr spec with placement smithi049=y;smithi129=x;count:2 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:20.379 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:20.394 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T11:38:20.394 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:20.394 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-17T11:38:20.394 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:20.394 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:20.395 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 11:38:00.417806051 +0000 2024-09-17T11:38:20.395 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.395 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.395 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T11:38:20.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: Reconfiguring mon.c (monmap changed)... 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: Reconfiguring daemon mon.c on smithi049 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi049=y;smithi129=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: Saving service mgr spec with placement smithi049=y;smithi129=x;count:2 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: Reconfiguring mon.c (monmap changed)... 2024-09-17T11:38:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: Reconfiguring daemon mon.c on smithi049 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi049=y;smithi129=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: Saving service mgr spec with placement smithi049=y;smithi129=x;count:2 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-17T11:38:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:20.458 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T11:38:20.458 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T11:38:20.458 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000198871 s, 2.6 MB/s 2024-09-17T11:38:20.460 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T11:38:20.516 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 11:38:00.418806030 +0000 2024-09-17T11:38:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.573 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.573 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 11:33:24.170743337 +0000 2024-09-17T11:38:20.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T11:38:20.635 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T11:38:20.635 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T11:38:20.635 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000179526 s, 2.9 MB/s 2024-09-17T11:38:20.637 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T11:38:20.693 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-09-17T11:38:20.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-17T11:38:20.942 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:38:21.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[30017]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:21.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[30017]: Reconfiguring mon.b (monmap changed)... 2024-09-17T11:38:21.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[30017]: Reconfiguring daemon mon.b on smithi129 2024-09-17T11:38:21.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[40223]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:21.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[40223]: Reconfiguring mon.b (monmap changed)... 2024-09-17T11:38:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:21 smithi049 ceph-mon[40223]: Reconfiguring daemon mon.b on smithi129 2024-09-17T11:38:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:21 smithi129 ceph-mon[33364]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:21 smithi129 ceph-mon[33364]: Reconfiguring mon.b (monmap changed)... 2024-09-17T11:38:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:21 smithi129 ceph-mon[33364]: Reconfiguring daemon mon.b on smithi129 2024-09-17T11:38:22.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[30017]: Deploying daemon mgr.x on smithi129 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.373 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:22 smithi049 ceph-mon[40223]: Deploying daemon mgr.x on smithi129 2024-09-17T11:38:22.636 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:22.637 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:22 smithi129 ceph-mon[33364]: Deploying daemon mgr.x on smithi129 2024-09-17T11:38:23.176 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 systemd[1]: Starting Ceph mgr.x for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:38:23.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:38:23.332 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:23 smithi049 ceph-mon[30017]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:23.332 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:23 smithi049 ceph-mon[40223]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:23.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-09-17T11:38:23.532 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:23 smithi129 ceph-mon[33364]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:23.533 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 podman[35013]: 2024-09-17 11:38:23.216352135 +0000 UTC m=+0.015567555 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T11:38:23.533 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 podman[35013]: 2024-09-17 11:38:23.3210444 +0000 UTC m=+0.120259851 container create 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=main-7498493, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T11:38:23.533 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 podman[35013]: 2024-09-17 11:38:23.532092996 +0000 UTC m=+0.331308410 container init 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=main-7498493, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T11:38:23.590 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:38:23.926 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 podman[35013]: 2024-09-17 11:38:23.537132464 +0000 UTC m=+0.336347873 container start 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=main-7498493) 2024-09-17T11:38:23.926 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 ceph-mgr[35051]: -- 172.21.15.129:0/1927498951 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c48ad850e0 con 0x55c48ad62800 2024-09-17T11:38:23.927 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 bash[35013]: 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 2024-09-17T11:38:23.927 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:38:23 smithi129 systemd[1]: Started Ceph mgr.x for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 -- 172.21.15.49:0/1584662309 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0103440 msgr2=0x7fcfc0109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 --2- 172.21.15.49:0/1584662309 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0103440 0x7fcfc0109dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009d00 tx=0x7fcfb802a910 comp rx=0 tx=0).stop 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 -- 172.21.15.49:0/1584662309 shutdown_connections 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 --2- 172.21.15.49:0/1584662309 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0103440 0x7fcfc0109dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 --2- 172.21.15.49:0/1584662309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0102a80 0x7fcfc0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 --2- 172.21.15.49:0/1584662309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0101920 0x7fcfc0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.013+0000 7fcfc7506640 1 -- 172.21.15.49:0/1584662309 >> 172.21.15.49:0/1584662309 conn(0x7fcfc00fd1d0 msgr2=0x7fcfc00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:24.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.014+0000 7fcfc7506640 1 -- 172.21.15.49:0/1584662309 shutdown_connections 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.014+0000 7fcfc7506640 1 -- 172.21.15.49:0/1584662309 wait complete. 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.014+0000 7fcfc7506640 1 Processor -- start 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.014+0000 7fcfc7506640 1 -- start start 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0102a80 0x7fcfc019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 0x7fcfc019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc010cc80 con 0x7fcfc0101920 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc010cb00 con 0x7fcfc0103440 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc010ce00 con 0x7fcfc0102a80 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49388/0 (socket says 172.21.15.49:49388) 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 -- 172.21.15.49:0/2781967008 learned_addr learned my addr 172.21.15.49:0/2781967008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:38:24.022 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc4aa1640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0102a80 0x7fcfc019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc5aa3640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 0x7fcfc019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 -- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0102a80 msgr2=0x7fcfc019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0102a80 0x7fcfc019d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 -- 172.21.15.49:0/2781967008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 msgr2=0x7fcfc019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 0x7fcfc019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 -- 172.21.15.49:0/2781967008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01a00f0 con 0x7fcfc0101920 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc5aa3640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 0x7fcfc019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc52a2640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d950 tx=0x7fcfbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc017030 con 0x7fcfc0101920 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fcfbc004020 con 0x7fcfc0101920 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01a03e0 con 0x7fcfc0101920 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.015+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0055a0 con 0x7fcfc0101920 2024-09-17T11:38:24.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.016+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a0970 con 0x7fcfc0101920 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.016+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 51786+0+0 (secure 0 0 0) 0x7fcfbc0047a0 con 0x7fcfc0101920 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.016+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0101d20 con 0x7fcfc0101920 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.017+0000 7fcfb27fc640 1 --2- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 0x7fcfac041e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.017+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (secure 0 0 0) 0x7fcfbc053cd0 con 0x7fcfc0101920 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.017+0000 7fcfc4aa1640 1 --2- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 0x7fcfac041e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.017+0000 7fcfc4aa1640 1 --2- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 0x7fcfac041e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40095f0 tx=0x7fcfb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:24.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.022+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fcfbc059020 con 0x7fcfc0101920 2024-09-17T11:38:24.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:24.216+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcfc0063590 con 0x7fcfac03f940 2024-09-17T11:38:24.874 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:24.875 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:24.876 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[30017]: Reconfiguring daemon mgr.y on smithi049 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T11:38:25.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:25.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:25.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:25.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:25 smithi049 ceph-mon[40223]: Reconfiguring daemon mgr.y on smithi049 2024-09-17T11:38:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:25 smithi129 ceph-mon[33364]: Reconfiguring daemon mgr.y on smithi049 2024-09-17T11:38:26.826 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:26.826 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:26.827 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"} : dispatch 2024-09-17T11:38:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"}]': finished 2024-09-17T11:38:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[30017]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T11:38:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"} : dispatch 2024-09-17T11:38:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"}]': finished 2024-09-17T11:38:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[40223]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T11:38:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:28 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:28 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"} : dispatch 2024-09-17T11:38:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1113245864' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a304fc7-f010-473e-ad18-e04e9b886490"}]': finished 2024-09-17T11:38:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:28 smithi129 ceph-mon[33364]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T11:38:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:28 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:29.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:29.124+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 14) ==== 102988+0+0 (secure 0 0 0) 0x7fcfbc004c70 con 0x7fcfc0101920 2024-09-17T11:38:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3835628618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: Standby manager daemon x started 2024-09-17T11:38:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:38:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3835628618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: Standby manager daemon x started 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:38:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:29 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3835628618' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: Standby manager daemon x started 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:38:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:29 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1361778652' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:38:30.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:30 smithi049 ceph-mon[30017]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-17T11:38:30.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:30 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:38:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:30 smithi049 ceph-mon[40223]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-17T11:38:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:30 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:38:30.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:30 smithi129 ceph-mon[33364]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-17T11:38:30.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:30 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:38:31.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:31 smithi049 ceph-mon[30017]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:31 smithi049 ceph-mon[40223]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:31 smithi129 ceph-mon[33364]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:33 smithi049 ceph-mon[30017]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:33.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:33 smithi049 ceph-mon[40223]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:33 smithi129 ceph-mon[33364]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[30017]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[40223]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-17T11:38:35.367 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:35 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:35 smithi129 ceph-mon[33364]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:35 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-17T11:38:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:35 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:36 smithi049 ceph-mon[30017]: Deploying daemon osd.0 on smithi049 2024-09-17T11:38:36.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:36 smithi049 ceph-mon[40223]: Deploying daemon osd.0 on smithi049 2024-09-17T11:38:36.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:36 smithi129 ceph-mon[33364]: Deploying daemon osd.0 on smithi049 2024-09-17T11:38:37.227 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:37 smithi049 ceph-mon[30017]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:37.228 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:37 smithi049 ceph-mon[40223]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:37.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:37 smithi129 ceph-mon[33364]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:39 smithi049 ceph-mon[30017]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:39 smithi049 ceph-mon[40223]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:39 smithi129 ceph-mon[33364]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[30017]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[40223]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:41.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:41 smithi129 ceph-mon[33364]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:43.280 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:43 smithi049 ceph-mon[40223]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:43.280 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:43 smithi049 ceph-mon[30017]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:43 smithi129 ceph-mon[33364]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.069 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-09-17T11:38:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.066+0000 7fcfb27fc640 1 -- 172.21.15.49:0/2781967008 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcfc0063590 con 0x7fcfac03f940 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.068+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 msgr2=0x7fcfac041e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 0x7fcfac041e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40095f0 tx=0x7fcfb4005f90 comp rx=0 tx=0).stop 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 msgr2=0x7fcfc006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d950 tx=0x7fcfbc00de10 comp rx=0 tx=0).stop 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 shutdown_connections 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcfac03f940 0x7fcfac041e00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc0103440 0x7fcfc019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcfc0102a80 0x7fcfc019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 --2- 172.21.15.49:0/2781967008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfc0101920 0x7fcfc006be00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 >> 172.21.15.49:0/2781967008 conn(0x7fcfc00fd1d0 msgr2=0x7fcfc00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 shutdown_connections 2024-09-17T11:38:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:45.069+0000 7fcfc7506640 1 -- 172.21.15.49:0/2781967008 wait complete. 2024-09-17T11:38:45.383 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[40223]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:45.383 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:45.383 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.383 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[30017]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:45.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:45.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:45 smithi129 ceph-mon[33364]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:45.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:38:45.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:45.662 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T11:38:45.664 INFO:tasks.cephadm:Deploying osd.1 on smithi049 with /dev/vg_nvme/lv_3... 2024-09-17T11:38:45.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-17T11:38:46.438 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:38:46.542 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:38:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T11:38:46.290+0000 7f94bf34c640 -1 osd.0 0 log_to_monitors true 2024-09-17T11:38:47.610 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:47 smithi049 ceph-mon[30017]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:47.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:47 smithi049 ceph-mon[30017]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-17T11:38:47.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:47 smithi049 ceph-mon[40223]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:47.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:47 smithi049 ceph-mon[40223]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-17T11:38:47.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:47 smithi129 ceph-mon[33364]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:47 smithi129 ceph-mon[33364]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-17T11:38:48.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[30017]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T11:38:48.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[30017]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T11:38:48.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[30017]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:38:48.578 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:48.578 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[40223]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T11:38:48.579 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[40223]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T11:38:48.579 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[40223]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:38:48.579 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:48 smithi129 ceph-mon[33364]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T11:38:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:48 smithi129 ceph-mon[33364]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T11:38:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:48 smithi129 ceph-mon[33364]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:38:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:48 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:48.900 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T11:38:48.574+0000 7f94bb2cf640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T11:38:48.900 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:38:48 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T11:38:48.577+0000 7f94b30ee640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:38:49.521 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:38:49.521 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:38:49.521 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:49.521 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:38:49.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' 2024-09-17T11:38:49.523 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:38:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:38:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: from='osd.0 v2:172.21.15.49:6801/501081646' entity='osd.0' 2024-09-17T11:38:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:50.798 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:38:50.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi049 2024-09-17T11:38:50.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:38:50.823 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 16064M 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: osd.0 v2:172.21.15.49:6801/501081646 boot 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:50.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[30017]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi049 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 16064M 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.825 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: osd.0 v2:172.21.15.49:6801/501081646 boot 2024-09-17T11:38:50.826 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T11:38:50.826 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:50.826 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:50 smithi049 ceph-mon[40223]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:50.827 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-09-17T11:38:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi049 2024-09-17T11:38:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 16064M 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: osd.0 v2:172.21.15.49:6801/501081646 boot 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:38:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:50 smithi129 ceph-mon[33364]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T11:38:51.082 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 -- 172.21.15.49:0/360118929 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf3006c090 msgr2=0x7fbf30105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/360118929 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf3006c090 0x7fbf30105600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009ca0 tx=0x7fbf2c02a8c0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 -- 172.21.15.49:0/360118929 shutdown_connections 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/360118929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf30108740 0x7fbf3010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/360118929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30105b90 0x7fbf30108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/360118929 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf3006c090 0x7fbf30105600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 -- 172.21.15.49:0/360118929 >> 172.21.15.49:0/360118929 conn(0x7fbf300fd1d0 msgr2=0x7fbf300ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 -- 172.21.15.49:0/360118929 shutdown_connections 2024-09-17T11:38:51.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.492+0000 7fbf37ba0640 1 -- 172.21.15.49:0/360118929 wait complete. 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 Processor -- start 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 -- start start 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 0x7fbf30199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3010dd40 con 0x7fbf3006c090 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3010dbc0 con 0x7fbf30108740 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf37ba0640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3010dec0 con 0x7fbf30105b90 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3593c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:51.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3593c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55350/0 (socket says 172.21.15.49:55350) 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:37344/0 (socket says 172.21.15.49:37344) 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3593c640 1 -- 172.21.15.49:0/1186927335 learned_addr learned my addr 172.21.15.49:0/1186927335 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3613d640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 0x7fbf30199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 -- 172.21.15.49:0/1186927335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 msgr2=0x7fbf30199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 0x7fbf30199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 -- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 msgr2=0x7fbf30198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3513b640 1 -- 172.21.15.49:0/1186927335 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3019a0b0 con 0x7fbf30105b90 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3593c640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.493+0000 7fbf3613d640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 0x7fbf30199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:38:51.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf3513b640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009990 tx=0x7fbf20009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20007840 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fbf20005020 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3019a3a0 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20004ae0 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.494+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301015b0 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.495+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf30105ff0 con 0x7fbf30105b90 2024-09-17T11:38:51.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.495+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 14) ==== 102988+0+0 (secure 0 0 0) 0x7fbf20004c80 con 0x7fbf30105b90 2024-09-17T11:38:51.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.496+0000 7fbf26ffd640 1 --2- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 0x7fbf1c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:38:51.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.496+0000 7fbf3593c640 1 --2- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 0x7fbf1c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:38:51.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.496+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1541+0+0 (secure 0 0 0) 0x7fbf2009ffe0 con 0x7fbf30105b90 2024-09-17T11:38:51.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.497+0000 7fbf3593c640 1 --2- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 0x7fbf1c07d110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009430 tx=0x7fbf2c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:38:51.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.501+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbf2006ca40 con 0x7fbf30105b90 2024-09-17T11:38:51.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:38:51.701+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbf3002e770 con 0x7fbf1c07ac50 2024-09-17T11:38:51.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:51 smithi049 ceph-mon[30017]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T11:38:51.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:51 smithi049 ceph-mon[40223]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T11:38:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:51 smithi129 ceph-mon[33364]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[30017]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[30017]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[40223]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[40223]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:52 smithi129 ceph-mon[33364]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:52 smithi129 ceph-mon[33364]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:38:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:38:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:38:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:38:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"} : dispatch 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"}]': finished 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2618055464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"} : dispatch 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"}]': finished 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T11:38:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:38:54.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2618055464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"} : dispatch 2024-09-17T11:38:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3044910353' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1d4bec5-5437-46c5-b724-270c2867bd33"}]': finished 2024-09-17T11:38:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T11:38:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:38:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2618055464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:38:57.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:56 smithi049 ceph-mon[30017]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:56 smithi049 ceph-mon[40223]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:56 smithi129 ceph-mon[33364]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:38:58 smithi049 ceph-mon[40223]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:38:58 smithi049 ceph-mon[30017]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:38:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:38:58 smithi129 ceph-mon[33364]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:01.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:00 smithi049 ceph-mon[30017]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:01.067 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:00 smithi049 ceph-mon[40223]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:00 smithi129 ceph-mon[33364]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:01.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-17T11:39:01.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:01.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[30017]: Deploying daemon osd.1 on smithi049 2024-09-17T11:39:01.995 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-17T11:39:01.995 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:01.996 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:01 smithi049 ceph-mon[40223]: Deploying daemon osd.1 on smithi049 2024-09-17T11:39:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-17T11:39:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:01 smithi129 ceph-mon[33364]: Deploying daemon osd.1 on smithi049 2024-09-17T11:39:02.886 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:02 smithi049 ceph-mon[40223]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:02.886 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:02 smithi049 ceph-mon[30017]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:03.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:02 smithi129 ceph-mon[33364]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:05.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:04 smithi049 ceph-mon[30017]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:05.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:04 smithi049 ceph-mon[40223]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:05.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:04 smithi129 ceph-mon[33364]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:06.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:06 smithi049 ceph-mon[30017]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:06.970 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:06 smithi049 ceph-mon[40223]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:06 smithi129 ceph-mon[33364]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:07 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:07 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:07 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:07 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:09 smithi129 ceph-mon[33364]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:09.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:09 smithi049 ceph-mon[30017]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:09.530 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:09 smithi049 ceph-mon[40223]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:10 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:10 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:10 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:10 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:10 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:10 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:10.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.963+0000 7fbf26ffd640 1 -- 172.21.15.49:0/1186927335 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf3002e770 con 0x7fbf1c07ac50 2024-09-17T11:39:10.965 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 1 on host 'smithi049' 2024-09-17T11:39:10.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 msgr2=0x7fbf1c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 0x7fbf1c07d110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009430 tx=0x7fbf2c009360 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 msgr2=0x7fbf30198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009990 tx=0x7fbf20009e50 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 shutdown_connections 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf1c07ac50 0x7fbf1c07d110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf30108740 0x7fbf30199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf30105b90 0x7fbf30198bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 --2- 172.21.15.49:0/1186927335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006c090 0x7fbf30198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.965+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 >> 172.21.15.49:0/1186927335 conn(0x7fbf300fd1d0 msgr2=0x7fbf30103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.966+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 shutdown_connections 2024-09-17T11:39:10.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:10.966+0000 7fbf37ba0640 1 -- 172.21.15.49:0/1186927335 wait complete. 2024-09-17T11:39:11.529 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[40223]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:11.529 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:11.529 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:11.529 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:11.529 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[30017]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:11 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:11.557 DEBUG:teuthology.orchestra.run.smithi049:osd.1> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T11:39:11.560 INFO:tasks.cephadm:Deploying osd.2 on smithi049 with /dev/vg_nvme/lv_2... 2024-09-17T11:39:11.560 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-17T11:39:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:11 smithi129 ceph-mon[33364]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:11 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:11 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:11 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:39:12.400 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:12 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T11:39:12.182+0000 7f08d66c2640 -1 osd.1 0 log_to_monitors true 2024-09-17T11:39:12.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:12 smithi049 ceph-mon[30017]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-17T11:39:12.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:12 smithi049 ceph-mon[40223]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-17T11:39:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:12 smithi129 ceph-mon[33364]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-17T11:39:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[30017]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[30017]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T11:39:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[30017]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T11:39:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[30017]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[40223]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[40223]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T11:39:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[40223]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T11:39:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[40223]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:13.651 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T11:39:13.450+0000 7f08d2645640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T11:39:13.651 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:13 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T11:39:13.456+0000 7f08ce46c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:39:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:13 smithi129 ceph-mon[33364]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:13 smithi129 ceph-mon[33364]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T11:39:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:13 smithi129 ceph-mon[33364]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T11:39:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:13 smithi129 ceph-mon[33364]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:13 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[40223]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[40223]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T11:39:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[30017]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[30017]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T11:39:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:14 smithi129 ceph-mon[33364]: from='osd.1 v2:172.21.15.49:6805/1599533781' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:14 smithi129 ceph-mon[33364]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T11:39:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: osd.1 v2:172.21.15.49:6805/1599533781 boot 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: osd.1 v2:172.21.15.49:6805/1599533781 boot 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T11:39:15.714 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:15.715 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:15 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: osd.1 v2:172.21.15.49:6805/1599533781 boot 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:15.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:15.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:15.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:15.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:15 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi049 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 8032M 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[30017]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi049 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 8032M 2024-09-17T11:39:16.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:16 smithi049 ceph-mon[40223]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T11:39:16.685 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:39:16.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-09-17T11:39:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:16 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi049 2024-09-17T11:39:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:16 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 8032M 2024-09-17T11:39:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:16 smithi129 ceph-mon[33364]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T11:39:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 -- 172.21.15.49:0/973918702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044104530 msgr2=0x7fe04410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 --2- 172.21.15.49:0/973918702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044104530 0x7fe04410aeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0098c0 tx=0x7fe03c02a390 comp rx=0 tx=0).stop 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 -- 172.21.15.49:0/973918702 shutdown_connections 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 --2- 172.21.15.49:0/973918702 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044104530 0x7fe04410aeb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 --2- 172.21.15.49:0/973918702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 0x7fe044103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:17.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 --2- 172.21.15.49:0/973918702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044102a10 0x7fe044102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 -- 172.21.15.49:0/973918702 >> 172.21.15.49:0/973918702 conn(0x7fe0440fe320 msgr2=0x7fe044100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 -- 172.21.15.49:0/973918702 shutdown_connections 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.474+0000 7fe04b17c640 1 -- 172.21.15.49:0/973918702 wait complete. 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 Processor -- start 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 -- start start 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 0x7fe044198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 0x7fe044199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe04410dd40 con 0x7fe044104530 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe04410dbc0 con 0x7fe044103b70 2024-09-17T11:39:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe04b17c640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fe04410dec0 con 0x7fe044102a10 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe048f18640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe048f18640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:34844/0 (socket says 172.21.15.49:34844) 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe048f18640 1 -- 172.21.15.49:0/3996194836 learned_addr learned my addr 172.21.15.49:0/3996194836 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe049719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 0x7fe044199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.475+0000 7fe043fff640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 0x7fe044198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 -- 172.21.15.49:0/3996194836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 msgr2=0x7fe044198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 0x7fe044198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 -- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 msgr2=0x7fe044199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 0x7fe044199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 -- 172.21.15.49:0/3996194836 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe044199fd0 con 0x7fe044102a10 2024-09-17T11:39:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe049719640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 0x7fe044199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe048f18640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e9c0 tx=0x7fe03400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03400cde0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fe0340035f0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04419a2c0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340044a0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.476+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441a09a0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.477+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe044102e10 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.478+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 14) ==== 102988+0+0 (secure 0 0 0) 0x7fe034004000 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.478+0000 7fe041ffb640 1 --2- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 0x7fe02407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.479+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 1833+0+0 (secure 0 0 0) 0x7fe03409cfd0 con 0x7fe044102a10 2024-09-17T11:39:17.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.479+0000 7fe043fff640 1 --2- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 0x7fe02407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.479+0000 7fe043fff640 1 --2- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 0x7fe02407d110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005fd0 tx=0x7fe02c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:39:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.483+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe034065550 con 0x7fe044102a10 2024-09-17T11:39:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:17 smithi049 ceph-mon[30017]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:17 smithi049 ceph-mon[40223]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:17.679+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe04402e820 con 0x7fe02407ac50 2024-09-17T11:39:17.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:17 smithi129 ceph-mon[33364]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:19.739 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:19 smithi049 ceph-mon[40223]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:19.739 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:19 smithi049 ceph-mon[40223]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:19.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:19 smithi049 ceph-mon[30017]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:19.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:19 smithi049 ceph-mon[30017]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:19 smithi129 ceph-mon[33364]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:19 smithi129 ceph-mon[33364]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:20.641 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"} : dispatch 2024-09-17T11:39:20.641 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"}]': finished 2024-09-17T11:39:20.641 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[40223]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T11:39:20.641 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:20.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"} : dispatch 2024-09-17T11:39:20.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"}]': finished 2024-09-17T11:39:20.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[30017]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T11:39:20.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"} : dispatch 2024-09-17T11:39:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2691233154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef100460-beb3-49ed-b03c-764b2bd5c333"}]': finished 2024-09-17T11:39:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:20 smithi129 ceph-mon[33364]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T11:39:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:21 smithi049 ceph-mon[30017]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3343009084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:21 smithi049 ceph-mon[40223]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3343009084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:21 smithi129 ceph-mon[33364]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3343009084' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:23.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:23 smithi049 ceph-mon[30017]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:23 smithi049 ceph-mon[40223]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:23 smithi129 ceph-mon[33364]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:25.573 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:25 smithi049 ceph-mon[30017]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:25.573 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:25 smithi049 ceph-mon[40223]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:25.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:25 smithi129 ceph-mon[33364]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[30017]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[40223]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-17T11:39:27.733 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:27 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:27 smithi129 ceph-mon[33364]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:27 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-17T11:39:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:27 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:28 smithi049 ceph-mon[30017]: Deploying daemon osd.2 on smithi049 2024-09-17T11:39:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:28 smithi049 ceph-mon[40223]: Deploying daemon osd.2 on smithi049 2024-09-17T11:39:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:28 smithi129 ceph-mon[33364]: Deploying daemon osd.2 on smithi049 2024-09-17T11:39:29.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:29 smithi049 ceph-mon[30017]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:29 smithi049 ceph-mon[40223]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:29.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:29 smithi129 ceph-mon[33364]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:31.772 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:31 smithi049 ceph-mon[30017]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:31.772 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:31 smithi049 ceph-mon[40223]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:31.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:31 smithi129 ceph-mon[33364]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:33.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[30017]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:33.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[40223]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:33.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:33.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:33 smithi129 ceph-mon[33364]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:33.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:33.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:33.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:35.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:35 smithi049 ceph-mon[30017]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:35.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:35 smithi049 ceph-mon[40223]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:35.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:35 smithi129 ceph-mon[33364]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:36.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.398+0000 7fe041ffb640 1 -- 172.21.15.49:0/3996194836 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe04402e820 con 0x7fe02407ac50 2024-09-17T11:39:36.401 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 2 on host 'smithi049' 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 msgr2=0x7fe02407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 0x7fe02407d110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005fd0 tx=0x7fe02c009870 comp rx=0 tx=0).stop 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 msgr2=0x7fe0441984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e9c0 tx=0x7fe03400ee80 comp rx=0 tx=0).stop 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 shutdown_connections 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe02407ac50 0x7fe02407d110 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:36.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe044104530 0x7fe044199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:36.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044103b70 0x7fe044198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:36.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 --2- 172.21.15.49:0/3996194836 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe044102a10 0x7fe0441984a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:36.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 >> 172.21.15.49:0/3996194836 conn(0x7fe0440fe320 msgr2=0x7fe0440ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:39:36.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 shutdown_connections 2024-09-17T11:39:36.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:36.401+0000 7fe04b17c640 1 -- 172.21.15.49:0/3996194836 wait complete. 2024-09-17T11:39:36.698 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.698 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.698 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:36.699 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.700 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:36.700 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:36.700 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.700 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:36 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:36 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:37.010 DEBUG:teuthology.orchestra.run.smithi049:osd.2> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T11:39:37.012 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_1... 2024-09-17T11:39:37.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-17T11:39:37.819 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:39:38.400 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:38 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T11:39:38.157+0000 7f34c012e640 -1 osd.2 0 log_to_monitors true 2024-09-17T11:39:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:38 smithi049 ceph-mon[30017]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:38 smithi049 ceph-mon[30017]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-17T11:39:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:38 smithi049 ceph-mon[40223]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:38 smithi049 ceph-mon[40223]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-17T11:39:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:38 smithi129 ceph-mon[33364]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:38 smithi129 ceph-mon[33364]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-17T11:39:40.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[30017]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[30017]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[30017]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[40223]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[40223]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[40223]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:40.069 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:39 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:39 smithi129 ceph-mon[33364]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T11:39:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:39 smithi129 ceph-mon[33364]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T11:39:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:39 smithi129 ceph-mon[33364]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:39:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:39 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:40.400 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T11:39:40.066+0000 7f34bc0b1640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T11:39:40.400 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T11:39:40.069+0000 7f34b3ed0640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:39:41.079 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[30017]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.080 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.081 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:40 smithi049 ceph-mon[40223]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' 2024-09-17T11:39:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:39:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T11:39:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:40 smithi129 ceph-mon[33364]: from='osd.2 v2:172.21.15.49:6809/185342269' entity='osd.2' 2024-09-17T11:39:41.949 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi049 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 5354M 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.950 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: osd.2 v2:172.21.15.49:6809/185342269 boot 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi049 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.951 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 5354M 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: osd.2 v2:172.21.15.49:6809/185342269 boot 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T11:39:41.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:41 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:39:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:39:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi049 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 5354M 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: osd.2 v2:172.21.15.49:6809/185342269 boot 2024-09-17T11:39:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T11:39:42.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:41 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:39:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:39:42.272 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-09-17T11:39:42.529 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 -- 172.21.15.49:0/1896893456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0101910 msgr2=0x7f96c0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 --2- 172.21.15.49:0/1896893456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0101910 0x7f96c0101d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009890 tx=0x7f96b002a380 comp rx=0 tx=0).stop 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 -- 172.21.15.49:0/1896893456 shutdown_connections 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 --2- 172.21.15.49:0/1896893456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0103430 0x7f96c0109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 --2- 172.21.15.49:0/1896893456 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 --2- 172.21.15.49:0/1896893456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0101910 0x7f96c0101d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 -- 172.21.15.49:0/1896893456 >> 172.21.15.49:0/1896893456 conn(0x7f96c00fd200 msgr2=0x7f96c00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:39:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 -- 172.21.15.49:0/1896893456 shutdown_connections 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.965+0000 7f96c4d72640 1 -- 172.21.15.49:0/1896893456 wait complete. 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 Processor -- start 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 -- start start 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0101910 0x7f96c0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96c010dd80 con 0x7f96c0103430 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96c010dc00 con 0x7f96c0101910 2024-09-17T11:39:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96c4d72640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f96c010df00 con 0x7f96c0102a70 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55480/0 (socket says 172.21.15.49:55480) 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:52254/0 (socket says 172.21.15.49:52254) 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96bed9d640 1 -- 172.21.15.49:0/3557199838 learned_addr learned my addr 172.21.15.49:0/3557199838 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.966+0000 7f96be59c640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0101910 0x7f96c0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 -- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 msgr2=0x7f96c0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 -- 172.21.15.49:0/3557199838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0101910 msgr2=0x7f96c0198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0101910 0x7f96c0198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 -- 172.21.15.49:0/3557199838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c019a0b0 con 0x7f96c0103430 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96bed9d640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96b800dac0 tx=0x7f96b800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8017070 con 0x7f96c0103430 2024-09-17T11:39:42.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c019a3a0 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f96b8003e40 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8004830 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.967+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a0a80 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.968+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c0101d10 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.969+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 14) ==== 102988+0+0 (secure 0 0 0) 0x7f96b80039c0 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.969+0000 7f96af7fe640 1 --2- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 0x7f96a807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.970+0000 7f96be59c640 1 --2- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 0x7f96a807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.970+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2125+0+0 (secure 0 0 0) 0x7f96b809d340 con 0x7f96c0103430 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.970+0000 7f96be59c640 1 --2- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 0x7f96a807d110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96b00303f0 tx=0x7f96b0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:39:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:42.974+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f96b8020020 con 0x7f96c0103430 2024-09-17T11:39:43.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:43.172+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f96c002e880 con 0x7f96a807ac50 2024-09-17T11:39:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[30017]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[30017]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T11:39:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[40223]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[40223]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T11:39:43.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:43 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:43 smithi129 ceph-mon[33364]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-17T11:39:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:43 smithi129 ceph-mon[33364]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T11:39:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:43 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:44.368 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:44.368 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T11:39:44.368 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:44.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T11:39:44.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:44.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:44.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:44.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:44.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:44.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:39:44.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59725]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59725]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59728]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59728]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.652 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59688]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.652 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59688]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59688]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59688]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59698]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59698]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59698]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59698]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59713]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.653 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59713]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59713]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59716]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59716]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59716]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59716]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.654 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59719]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.655 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59719]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.655 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.655 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.655 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59722]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.655 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59722]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59731]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59731]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59731]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59746]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:44 smithi049 sudo[59746]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35493]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T11:39:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35493]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35493]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35493]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35496]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T11:39:44.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T11:39:44.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35496]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T11:39:44.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:44 smithi129 sudo[35496]: pam_unix(sudo:session): session closed for user root 2024-09-17T11:39:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[30017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:45 smithi049 ceph-mon[40223]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:45.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:45.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T11:39:45.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:39:45.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:39:45.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:39:45.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:45 smithi129 ceph-mon[33364]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T11:39:46.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:39:46.397+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f96b8061520 con 0x7f96c0103430 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"} : dispatch 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"}]': finished 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/604787567' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T11:39:46.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"} : dispatch 2024-09-17T11:39:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"}]': finished 2024-09-17T11:39:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T11:39:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:39:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/604787567' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: osdmap e22: 3 total, 3 up, 3 in 2024-09-17T11:39:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"} : dispatch 2024-09-17T11:39:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1193845475' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5"}]': finished 2024-09-17T11:39:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T11:39:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:39:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/604787567' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:39:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[40223]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[40223]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:39:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[40223]: Cluster is now healthy 2024-09-17T11:39:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[40223]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T11:39:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[30017]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[30017]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:39:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[30017]: Cluster is now healthy 2024-09-17T11:39:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:47 smithi049 ceph-mon[30017]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T11:39:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:47 smithi129 ceph-mon[33364]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:47 smithi129 ceph-mon[33364]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:39:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:47 smithi129 ceph-mon[33364]: Cluster is now healthy 2024-09-17T11:39:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:47 smithi129 ceph-mon[33364]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T11:39:49.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:49 smithi049 ceph-mon[30017]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:49.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:49 smithi049 ceph-mon[40223]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:49 smithi129 ceph-mon[33364]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:51.276 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:51 smithi049 ceph-mon[40223]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:51.276 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:51 smithi049 ceph-mon[30017]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:51.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:51 smithi129 ceph-mon[33364]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[30017]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[40223]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-17T11:39:53.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:53 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:53.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:53 smithi129 ceph-mon[33364]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:53.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:53 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-17T11:39:53.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:53 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:39:54.388 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:54 smithi049 ceph-mon[40223]: Deploying daemon osd.3 on smithi049 2024-09-17T11:39:54.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:54 smithi049 ceph-mon[30017]: Deploying daemon osd.3 on smithi049 2024-09-17T11:39:54.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:54 smithi129 ceph-mon[33364]: Deploying daemon osd.3 on smithi049 2024-09-17T11:39:55.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:55 smithi049 ceph-mon[30017]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:55.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:55 smithi049 ceph-mon[40223]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:55.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:55 smithi129 ceph-mon[33364]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:57.222 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:57 smithi049 ceph-mon[40223]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:57.222 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:57 smithi049 ceph-mon[30017]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:57.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:57 smithi129 ceph-mon[33364]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[30017]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[40223]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:39:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:59 smithi129 ceph-mon[33364]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:39:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:39:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:39:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:39:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:00 smithi049 ceph-mon[40223]: overall HEALTH_OK 2024-09-17T11:40:00.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:00 smithi049 ceph-mon[30017]: overall HEALTH_OK 2024-09-17T11:40:00.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:00 smithi129 ceph-mon[33364]: overall HEALTH_OK 2024-09-17T11:40:01.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[30017]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:01.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[40223]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:01.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:01 smithi129 ceph-mon[33364]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:01.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.522+0000 7f96af7fe640 1 -- 172.21.15.49:0/3557199838 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f96c002e880 con 0x7f96a807ac50 2024-09-17T11:40:01.525 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 msgr2=0x7f96a807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 0x7f96a807d110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96b00303f0 tx=0x7f96b0005d30 comp rx=0 tx=0).stop 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 msgr2=0x7f96c0199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96b800dac0 tx=0x7f96b800df80 comp rx=0 tx=0).stop 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 shutdown_connections 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> v2:172.21.15.49:6800/1112797402 conn(0x7f96a807ac50 0x7f96a807d110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96c0103430 0x7f96c0199810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f96c0102a70 0x7f96c0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 --2- 172.21.15.49:0/3557199838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96c0101910 0x7f96c0198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 >> 172.21.15.49:0/3557199838 conn(0x7f96c00fd200 msgr2=0x7f96c00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 shutdown_connections 2024-09-17T11:40:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:40:01.525+0000 7f96c4d72640 1 -- 172.21.15.49:0/3557199838 wait complete. 2024-09-17T11:40:02.154 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.3.service 2024-09-17T11:40:02.156 INFO:tasks.cephadm:Deploying osd.4 on smithi129 with /dev/vg_nvme/lv_4... 2024-09-17T11:40:02.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.416 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:03.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:03 smithi049 ceph-mon[30017]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:03.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:03 smithi049 ceph-mon[40223]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:03.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:03 smithi129 ceph-mon[33364]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:03.579 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 11:40:03 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T11:40:03.377+0000 7ff213369640 -1 osd.3 0 log_to_monitors true 2024-09-17T11:40:04.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:04 smithi049 ceph-mon[30017]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:04 smithi049 ceph-mon[30017]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:04 smithi049 ceph-mon[40223]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.399 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:04 smithi049 ceph-mon[40223]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:04 smithi129 ceph-mon[33364]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:04 smithi129 ceph-mon[33364]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-17T11:40:04.852 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:40:04.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-09-17T11:40:05.117 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:05.400 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T11:40:05.133+0000 7ff20f2ec640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T11:40:05.400 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T11:40:05.137+0000 7ff20b113640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:40:05.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:05.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:05.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:05.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:05 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T11:40:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T11:40:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='osd.3 v2:172.21.15.49:6813/1277715496' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:05 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 -- 172.21.15.129:0/2649475394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354103b60 msgr2=0x7fb354103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/2649475394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354103b60 0x7fb354103fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb344009930 tx=0x7fb34402a420 comp rx=0 tx=0).stop 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 -- 172.21.15.129:0/2649475394 shutdown_connections 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/2649475394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354104520 0x7fb35410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/2649475394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354103b60 0x7fb354103fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/2649475394 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354102a00 0x7fb354102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 -- 172.21.15.129:0/2649475394 >> 172.21.15.129:0/2649475394 conn(0x7fb3540fe2f0 msgr2=0x7fb354100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 -- 172.21.15.129:0/2649475394 shutdown_connections 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.568+0000 7fb35bb2d640 1 -- 172.21.15.129:0/2649475394 wait complete. 2024-09-17T11:40:05.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 Processor -- start 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 -- start start 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354102a00 0x7fb354198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354103b60 0x7fb354198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb35410de60 con 0x7fb354102a00 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb35410dce0 con 0x7fb354104520 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fb35410dfe0 con 0x7fb354103b60 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39690/0 (socket says 172.21.15.129:39690) 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 -- 172.21.15.129:0/900921074 learned_addr learned my addr 172.21.15.129:0/900921074 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:40:05.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb3598c9640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354102a00 0x7fb354198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb3590c8640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354103b60 0x7fb354198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 -- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354103b60 msgr2=0x7fb354198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354103b60 0x7fb354198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 -- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354102a00 msgr2=0x7fb354198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354102a00 0x7fb354198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 -- 172.21.15.129:0/900921074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35419a190 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35a0ca640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00b970 tx=0x7fb34c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c019070 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fb34c014440 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0075d0 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.569+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35419a480 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.570+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3541a0b60 con 0x7fb354104520 2024-09-17T11:40:05.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.571+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb354102e80 con 0x7fb354104520 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.571+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7fb34c01c070 con 0x7fb354104520 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.572+0000 7fb34affd640 1 --2- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 0x7fb34007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.572+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 2663+0+0 (secure 0 0 0) 0x7fb34c09c550 con 0x7fb354104520 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.572+0000 7fb3598c9640 1 --2- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 0x7fb34007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.572+0000 7fb3598c9640 1 --2- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 0x7fb34007d1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb350009890 tx=0x7fb350008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:05.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.576+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb34c068b10 con 0x7fb354104520 2024-09-17T11:40:05.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:05.759+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb35402e820 con 0x7fb34007ace0 2024-09-17T11:40:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi049 2024-09-17T11:40:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 4016M 2024-09-17T11:40:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:40:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T11:40:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi049 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 4016M 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:40:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi049 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 4016M 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: osdmap e25: 4 total, 3 up, 4 in 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:40:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:40:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: osd.3 v2:172.21.15.49:6813/1277715496 boot 2024-09-17T11:40:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T11:40:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:07 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: osd.3 v2:172.21.15.49:6813/1277715496 boot 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T11:40:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: osd.3 v2:172.21.15.49:6813/1277715496 boot 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T11:40:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:07 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:40:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:08 smithi049 ceph-mon[30017]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T11:40:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:08 smithi049 ceph-mon[40223]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T11:40:08.400 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:08 smithi129 ceph-mon[33364]: osdmap e27: 4 total, 4 up, 4 in 2024-09-17T11:40:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/2046978431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"}]': finished 2024-09-17T11:40:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T11:40:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/1006971245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/2046978431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"}]': finished 2024-09-17T11:40:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/1006971245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/2046978431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"} : dispatch 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9500ec09-085d-45ae-8356-ea20c1a7d064"}]': finished 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/1006971245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:11 smithi129 ceph-mon[33364]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:11 smithi049 ceph-mon[30017]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:11.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:11 smithi049 ceph-mon[40223]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:13.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:13 smithi129 ceph-mon[33364]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:13 smithi049 ceph-mon[30017]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:13 smithi049 ceph-mon[40223]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:15.237 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:15 smithi129 ceph-mon[33364]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:15.452 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:15 smithi049 ceph-mon[40223]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:15.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:15 smithi049 ceph-mon[30017]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-17T11:40:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:16 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:16 smithi129 ceph-mon[33364]: Deploying daemon osd.4 on smithi129 2024-09-17T11:40:16.469 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-17T11:40:16.469 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:16.470 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[40223]: Deploying daemon osd.4 on smithi129 2024-09-17T11:40:16.470 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-17T11:40:16.470 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:16.470 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:16 smithi049 ceph-mon[30017]: Deploying daemon osd.4 on smithi129 2024-09-17T11:40:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:17 smithi129 ceph-mon[33364]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:17 smithi049 ceph-mon[30017]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:17 smithi049 ceph-mon[40223]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:19 smithi129 ceph-mon[33364]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:19.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:19 smithi049 ceph-mon[30017]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:19 smithi049 ceph-mon[40223]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:21 smithi129 ceph-mon[33364]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:21 smithi049 ceph-mon[30017]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:21.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:21 smithi049 ceph-mon[40223]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:22.414 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:22.414 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:22.414 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:22 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:22 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:23.368 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:23 smithi129 ceph-mon[33364]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:23 smithi049 ceph-mon[30017]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:23 smithi049 ceph-mon[40223]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:25.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.840+0000 7fb34affd640 1 -- 172.21.15.129:0/900921074 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb35402e820 con 0x7fb34007ace0 2024-09-17T11:40:25.842 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 4 on host 'smithi129' 2024-09-17T11:40:25.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.842+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 msgr2=0x7fb34007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:25.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.842+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 0x7fb34007d1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb350009890 tx=0x7fb350008040 comp rx=0 tx=0).stop 2024-09-17T11:40:25.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.842+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 msgr2=0x7fb3541998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:25.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.842+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00b970 tx=0x7fb34c00be30 comp rx=0 tx=0).stop 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 shutdown_connections 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb34007ace0 0x7fb34007d1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb354104520 0x7fb3541998f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb354103b60 0x7fb354198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 --2- 172.21.15.129:0/900921074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb354102a00 0x7fb354198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 >> 172.21.15.129:0/900921074 conn(0x7fb3540fe2f0 msgr2=0x7fb3540ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 shutdown_connections 2024-09-17T11:40:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:25.843+0000 7fb35bb2d640 1 -- 172.21.15.129:0/900921074 wait complete. 2024-09-17T11:40:26.480 DEBUG:teuthology.orchestra.run.smithi129:osd.4> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.4.service 2024-09-17T11:40:26.482 INFO:tasks.cephadm:Deploying osd.5 on smithi129 with /dev/vg_nvme/lv_3... 2024-09-17T11:40:26.482 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-17T11:40:26.730 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T11:40:26.649+0000 7ff47ab4c640 -1 osd.4 0 log_to_monitors true 2024-09-17T11:40:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[30017]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:26 smithi049 ceph-mon[40223]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.162 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:26 smithi129 ceph-mon[33364]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-17T11:40:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:27 smithi129 ceph-mon[33364]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T11:40:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:27 smithi129 ceph-mon[33364]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T11:40:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:27 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:27 smithi129 ceph-mon[33364]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:27 smithi129 ceph-mon[33364]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[30017]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T11:40:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[30017]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T11:40:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[30017]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[30017]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[40223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T11:40:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[40223]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T11:40:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[40223]: from='osd.4 v2:172.21.15.129:6800/2156637849' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:27 smithi049 ceph-mon[40223]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:28.650 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T11:40:28.304+0000 7ff476acf640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T11:40:28.650 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T11:40:28.307+0000 7ff46e8ee640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:40:28.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:28.910 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:28.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T11:40:28.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:28.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:28.911 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:28 smithi129 ceph-mon[33364]: from='osd.4 ' entity='osd.4' 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[30017]: from='osd.4 ' entity='osd.4' 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: osdmap e30: 5 total, 4 up, 5 in 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:28 smithi049 ceph-mon[40223]: from='osd.4 ' entity='osd.4' 2024-09-17T11:40:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:40:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:40:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: osd.4 v2:172.21.15.129:6800/2156637849 boot 2024-09-17T11:40:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T11:40:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:29 smithi129 ceph-mon[33364]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: osd.4 v2:172.21.15.129:6800/2156637849 boot 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[40223]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T11:40:30.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:40:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:40:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: osd.4 v2:172.21.15.129:6800/2156637849 boot 2024-09-17T11:40:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T11:40:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:40:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:29 smithi049 ceph-mon[30017]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi129 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:31.422 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-17T11:40:31.423 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.423 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:31.423 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:31.423 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:31 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T11:40:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi129 2024-09-17T11:40:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T11:40:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi129 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:31.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:31 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:31.726 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:40:31.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-09-17T11:40:31.996 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 -- 172.21.15.129:0/642507696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 msgr2=0x7f7410109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 --2- 172.21.15.129:0/642507696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f7410109d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7408009cd0 tx=0x7f740802a8e0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 -- 172.21.15.129:0/642507696 shutdown_connections 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 --2- 172.21.15.129:0/642507696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f7410109d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 --2- 172.21.15.129:0/642507696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f7410102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 --2- 172.21.15.129:0/642507696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 0x7f7410101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 -- 172.21.15.129:0/642507696 >> 172.21.15.129:0/642507696 conn(0x7f74100fd1b0 msgr2=0x7f74100ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 -- 172.21.15.129:0/642507696 shutdown_connections 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.433+0000 7f7414850640 1 -- 172.21.15.129:0/642507696 wait complete. 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 Processor -- start 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 -- start start 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 0x7f741006b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f741019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:32.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f741010cc10 con 0x7f74101018e0 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f741010ca90 con 0x7f7410103400 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7414850640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f741010cd90 con 0x7f7410102a40 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48536/0 (socket says 172.21.15.129:48536) 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 -- 172.21.15.129:0/1773819942 learned_addr learned my addr 172.21.15.129:0/1773819942 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740e59c640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 0x7f741006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f7407fff640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f741019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 -- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 msgr2=0x7f741019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f741019d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 -- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 msgr2=0x7f741006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 0x7f741006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.434+0000 7f740ed9d640 1 -- 172.21.15.129:0/1773819942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74101a0050 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f7407fff640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f741019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f740ed9d640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f74080097b0 tx=0x7f7408003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004450 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a02e0 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a0870 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f7408031430 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.435+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408046480 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.436+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f00050b0 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.437+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f74080045f0 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.437+0000 7f74077fe640 1 --2- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 0x7f73fc07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.437+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 3205+0+0 (secure 0 0 0) 0x7f740802e030 con 0x7f7410103400 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.437+0000 7f740e59c640 1 --2- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 0x7f73fc07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:32.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.438+0000 7f740e59c640 1 --2- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 0x7f73fc07d1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7400009bd0 tx=0x7f7400005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:32.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.441+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f740804b060 con 0x7f7410103400 2024-09-17T11:40:32.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:32.626+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f73f0000bf0 con 0x7f73fc07ace0 2024-09-17T11:40:33.286 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:33 smithi129 ceph-mon[33364]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T11:40:33.287 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:33.287 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:33.287 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[30017]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T11:40:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[40223]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T11:40:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:34.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:34 smithi129 ceph-mon[33364]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:34.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:34 smithi049 ceph-mon[30017]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:34.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:34 smithi049 ceph-mon[40223]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:40:35.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-17T11:40:35.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/2375962569' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"}]': finished 2024-09-17T11:40:35.608 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: osdmap e34: 6 total, 5 up, 6 in 2024-09-17T11:40:35.609 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:35 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:35.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-17T11:40:35.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/2375962569' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"}]': finished 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: osdmap e34: 6 total, 5 up, 6 in 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/2375962569' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"} : dispatch 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9691398-5234-4fb1-ba46-1369ac2b43cb"}]': finished 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: osdmap e34: 6 total, 5 up, 6 in 2024-09-17T11:40:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:35 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:36.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/1164595833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:36.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/1164595833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:36.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/1164595833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:40:37.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:37 smithi049 ceph-mon[30017]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T11:40:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:37 smithi049 ceph-mon[40223]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T11:40:37.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:37 smithi129 ceph-mon[33364]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-17T11:40:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:39 smithi129 ceph-mon[33364]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T11:40:39.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:39 smithi049 ceph-mon[30017]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T11:40:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:39 smithi049 ceph-mon[40223]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T11:40:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:41 smithi129 ceph-mon[33364]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-17T11:40:41.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:41 smithi049 ceph-mon[30017]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-17T11:40:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:41 smithi049 ceph-mon[40223]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-17T11:40:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:43 smithi129 ceph-mon[33364]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:43 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-17T11:40:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:43 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:43 smithi129 ceph-mon[33364]: Deploying daemon osd.5 on smithi129 2024-09-17T11:40:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[30017]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-17T11:40:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[30017]: Deploying daemon osd.5 on smithi129 2024-09-17T11:40:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[40223]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-17T11:40:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:43 smithi049 ceph-mon[40223]: Deploying daemon osd.5 on smithi129 2024-09-17T11:40:45.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:45 smithi129 ceph-mon[33364]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:45.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:45 smithi049 ceph-mon[30017]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:45 smithi049 ceph-mon[40223]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:47.484 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:47 smithi129 ceph-mon[33364]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:47 smithi049 ceph-mon[30017]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:47 smithi049 ceph-mon[40223]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:49 smithi129 ceph-mon[33364]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:49.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:49.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:49 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[30017]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[40223]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:49.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:49 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:51.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:51 smithi129 ceph-mon[33364]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:51 smithi049 ceph-mon[30017]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:51 smithi049 ceph-mon[40223]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.187+0000 7f74077fe640 1 -- 172.21.15.129:0/1773819942 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f73f0000bf0 con 0x7f73fc07ace0 2024-09-17T11:40:52.189 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 5 on host 'smithi129' 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 msgr2=0x7f73fc07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 0x7f73fc07d1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7400009bd0 tx=0x7f7400005f90 comp rx=0 tx=0).stop 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 msgr2=0x7f741019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f74080097b0 tx=0x7f7408003f10 comp rx=0 tx=0).stop 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 shutdown_connections 2024-09-17T11:40:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> v2:172.21.15.49:6800/1112797402 conn(0x7f73fc07ace0 0x7f73fc07d1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7410103400 0x7f741019f790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7410102a40 0x7f741019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 --2- 172.21.15.129:0/1773819942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74101018e0 0x7f741006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 >> 172.21.15.129:0/1773819942 conn(0x7f74100fd1b0 msgr2=0x7f74100fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 shutdown_connections 2024-09-17T11:40:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:52.190+0000 7f7414850640 1 -- 172.21.15.129:0/1773819942 wait complete. 2024-09-17T11:40:52.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:52.975 DEBUG:teuthology.orchestra.run.smithi129:osd.5> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.5.service 2024-09-17T11:40:52.978 INFO:tasks.cephadm:Deploying osd.6 on smithi129 with /dev/vg_nvme/lv_2... 2024-09-17T11:40:52.978 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-17T11:40:53.454 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:53 smithi129 ceph-mon[33364]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:53.454 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:53 smithi129 ceph-mon[33364]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:53.454 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:53 smithi129 ceph-mon[33364]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:53.454 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 11:40:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T11:40:53.225+0000 7f9dc43f7640 -1 osd.5 0 log_to_monitors true 2024-09-17T11:40:53.763 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[30017]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[30017]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[30017]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[40223]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[40223]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:53.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:53 smithi049 ceph-mon[40223]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-17T11:40:54.638 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-mon[33364]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T11:40:54.638 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-mon[33364]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T11:40:54.638 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:54.638 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-mon[33364]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.638 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-mon[33364]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[30017]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T11:40:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[30017]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T11:40:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[30017]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[30017]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[40223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T11:40:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[40223]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T11:40:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[40223]: from='osd.5 v2:172.21.15.129:6804/2449624152' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:54 smithi049 ceph-mon[40223]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:40:54.926 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T11:40:54.839+0000 7f9dc0b8e640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T11:40:54.926 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 11:40:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T11:40:54.843+0000 7f9db8199640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.595 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:55 smithi129 ceph-mon[33364]: from='osd.5 ' entity='osd.5' 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[30017]: from='osd.5 ' entity='osd.5' 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:55 smithi049 ceph-mon[40223]: from='osd.5 ' entity='osd.5' 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: osd.5 v2:172.21.15.129:6804/2449624152 boot 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: osdmap e37: 6 total, 6 up, 6 in 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:56.457 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:56 smithi129 ceph-mon[33364]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: osd.5 v2:172.21.15.129:6804/2449624152 boot 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: osdmap e37: 6 total, 6 up, 6 in 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[30017]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T11:40:56.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:40:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:40:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: osd.5 v2:172.21.15.129:6804/2449624152 boot 2024-09-17T11:40:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: osdmap e37: 6 total, 6 up, 6 in 2024-09-17T11:40:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:40:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:56 smithi049 ceph-mon[40223]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T11:40:57.872 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: pgmap v129: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:57.872 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi129 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:57 smithi129 ceph-mon[33364]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T11:40:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: pgmap v129: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi129 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[30017]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: pgmap v129: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi129 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:40:57.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:57 smithi049 ceph-mon[40223]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T11:40:57.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-09-17T11:40:58.147 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 -- 172.21.15.129:0/893073175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248104510 msgr2=0x7f924810ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 --2- 172.21.15.129:0/893073175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248104510 0x7f924810ae90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9240009cd0 tx=0x7f924002a8e0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 -- 172.21.15.129:0/893073175 shutdown_connections 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 --2- 172.21.15.129:0/893073175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248104510 0x7f924810ae90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 --2- 172.21.15.129:0/893073175 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9248103b50 0x7f9248103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 --2- 172.21.15.129:0/893073175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92481029f0 0x7f9248102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 -- 172.21.15.129:0/893073175 >> 172.21.15.129:0/893073175 conn(0x7f92480fe2c0 msgr2=0x7f9248100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 -- 172.21.15.129:0/893073175 shutdown_connections 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.608+0000 7f924c8cd640 1 -- 172.21.15.129:0/893073175 wait complete. 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.609+0000 7f924c8cd640 1 Processor -- start 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.609+0000 7f924c8cd640 1 -- start start 2024-09-17T11:40:58.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 0x7f92481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248104510 0x7f9248199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f924810dda0 con 0x7f9248104510 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f924810dc20 con 0x7f9248103b50 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f924810df20 con 0x7f92481029f0 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50864/0 (socket says 172.21.15.129:50864) 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 -- 172.21.15.129:0/1349490869 learned_addr learned my addr 172.21.15.129:0/1349490869 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9246d9d640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248104510 0x7f9248199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924659c640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 0x7f92481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 -- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 msgr2=0x7f92481984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 0x7f92481984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 -- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248104510 msgr2=0x7f9248199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248104510 0x7f9248199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 -- 172.21.15.129:0/1349490869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9248199fd0 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924659c640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 0x7f92481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f9245d9b640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f923000b4a0 tx=0x7f923000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230014070 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f9230010a90 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819a2c0 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230017b80 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.610+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a09a0 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.611+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9248102df0 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.612+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f9230017070 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.613+0000 7f923f7fe640 1 --2- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 0x7f923407d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.613+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 3497+0+0 (secure 0 0 0) 0x7f923009ca10 con 0x7f9248103b50 2024-09-17T11:40:58.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.613+0000 7f924659c640 1 --2- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 0x7f923407d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:40:58.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.614+0000 7f924659c640 1 --2- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 0x7f923407d1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f923800baa0 tx=0x7f9238005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:40:58.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.617+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f92300648b0 con 0x7f9248103b50 2024-09-17T11:40:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:40:58.801+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f924802e930 con 0x7f923407ace0 2024-09-17T11:40:59.884 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:59 smithi129 ceph-mon[33364]: pgmap v132: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:59.884 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:59.884 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:59.884 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:40:59 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[30017]: pgmap v132: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[40223]: pgmap v132: 1 pgs: 1 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:40:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:40:59 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:00.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:00 smithi129 ceph-mon[33364]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:00 smithi049 ceph-mon[30017]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:00 smithi049 ceph-mon[40223]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: pgmap v133: 1 pgs: 1 unknown; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/903482301' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"}]': finished 2024-09-17T11:41:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: osdmap e40: 7 total, 6 up, 7 in 2024-09-17T11:41:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: pgmap v133: 1 pgs: 1 unknown; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/903482301' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"}]': finished 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: osdmap e40: 7 total, 6 up, 7 in 2024-09-17T11:41:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: pgmap v133: 1 pgs: 1 unknown; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/903482301' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"} : dispatch 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34249b34-b42f-4289-9797-11f3e971069e"}]': finished 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: osdmap e40: 7 total, 6 up, 7 in 2024-09-17T11:41:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:02.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/1209098072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/1209098072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/1209098072' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:03 smithi049 ceph-mon[30017]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T11:41:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:03 smithi049 ceph-mon[40223]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T11:41:03.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:03 smithi129 ceph-mon[33364]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T11:41:05.675 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:05 smithi129 ceph-mon[33364]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T11:41:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:05 smithi049 ceph-mon[30017]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T11:41:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:05 smithi049 ceph-mon[40223]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-17T11:41:07.665 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:07 smithi129 ceph-mon[33364]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-17T11:41:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:07 smithi049 ceph-mon[30017]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-17T11:41:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:07 smithi049 ceph-mon[40223]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-17T11:41:08.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-17T11:41:08.862 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:08.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-17T11:41:08.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-17T11:41:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:09.885 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:09 smithi129 ceph-mon[33364]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:09.885 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:09 smithi129 ceph-mon[33364]: Deploying daemon osd.6 on smithi129 2024-09-17T11:41:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:09 smithi049 ceph-mon[30017]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:09 smithi049 ceph-mon[30017]: Deploying daemon osd.6 on smithi129 2024-09-17T11:41:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:09 smithi049 ceph-mon[40223]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:09 smithi049 ceph-mon[40223]: Deploying daemon osd.6 on smithi129 2024-09-17T11:41:11.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:11 smithi129 ceph-mon[33364]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:11 smithi049 ceph-mon[30017]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:11.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:11 smithi049 ceph-mon[40223]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T11:41:13.892 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:13 smithi129 ceph-mon[33364]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:13 smithi049 ceph-mon[30017]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:13 smithi049 ceph-mon[40223]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:14 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:14 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:15.804 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:15 smithi129 ceph-mon[33364]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:15.804 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:15 smithi049 ceph-mon[40223]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:15.805 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:15 smithi049 ceph-mon[30017]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:16.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:16 smithi049 ceph-mon[30017]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:16 smithi049 ceph-mon[40223]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:16 smithi129 ceph-mon[33364]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:17.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.615+0000 7f923f7fe640 1 -- 172.21.15.129:0/1349490869 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f924802e930 con 0x7f923407ace0 2024-09-17T11:41:17.617 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 6 on host 'smithi129' 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 msgr2=0x7f923407d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 0x7f923407d1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f923800baa0 tx=0x7f9238005d90 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 msgr2=0x7f9248198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f923000b4a0 tx=0x7f923000b960 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 shutdown_connections 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> v2:172.21.15.49:6800/1112797402 conn(0x7f923407ace0 0x7f923407d1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248104510 0x7f9248199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9248103b50 0x7f9248198af0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 --2- 172.21.15.129:0/1349490869 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f92481029f0 0x7f92481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 >> 172.21.15.129:0/1349490869 conn(0x7f92480fe2c0 msgr2=0x7f92480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 shutdown_connections 2024-09-17T11:41:17.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:17.618+0000 7f924c8cd640 1 -- 172.21.15.129:0/1349490869 wait complete. 2024-09-17T11:41:18.317 DEBUG:teuthology.orchestra.run.smithi129:osd.6> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.6.service 2024-09-17T11:41:18.319 INFO:tasks.cephadm:Deploying osd.7 on smithi129 with /dev/vg_nvme/lv_1... 2024-09-17T11:41:18.319 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-17T11:41:18.636 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:18.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:18 smithi129 ceph-mon[33364]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:18.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.873 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:18 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[40223]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[30017]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:18 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:19.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:19 smithi129 ceph-mon[33364]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:19.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:19 smithi129 ceph-mon[33364]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:19.714 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 11:41:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T11:41:19.434+0000 7f3d11633640 -1 osd.6 0 log_to_monitors true 2024-09-17T11:41:19.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:19 smithi049 ceph-mon[30017]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:19.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:19 smithi049 ceph-mon[30017]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:19 smithi049 ceph-mon[40223]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:19 smithi049 ceph-mon[40223]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-17T11:41:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T11:41:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: osdmap e41: 7 total, 6 up, 7 in 2024-09-17T11:41:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[30017]: osdmap e42: 7 total, 6 up, 7 in 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: osdmap e41: 7 total, 6 up, 7 in 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:20.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:20 smithi049 ceph-mon[40223]: osdmap e42: 7 total, 6 up, 7 in 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: from='osd.6 v2:172.21.15.129:6808/3044459329' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: osdmap e41: 7 total, 6 up, 7 in 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-mon[33364]: osdmap e42: 7 total, 6 up, 7 in 2024-09-17T11:41:20.927 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T11:41:20.621+0000 7f3d0d5b6640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T11:41:20.927 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 11:41:20 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T11:41:20.628+0000 7f3d093dd640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:41:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[30017]: osd.6 v2:172.21.15.129:6808/3044459329 boot 2024-09-17T11:41:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[30017]: osdmap e43: 7 total, 7 up, 7 in 2024-09-17T11:41:21.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[40223]: osd.6 v2:172.21.15.129:6808/3044459329 boot 2024-09-17T11:41:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[40223]: osdmap e43: 7 total, 7 up, 7 in 2024-09-17T11:41:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:21 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:21 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:21 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:21 smithi129 ceph-mon[33364]: osd.6 v2:172.21.15.129:6808/3044459329 boot 2024-09-17T11:41:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:21 smithi129 ceph-mon[33364]: osdmap e43: 7 total, 7 up, 7 in 2024-09-17T11:41:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:21 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:41:22.023 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T11:41:22.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-09-17T11:41:22.637 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:41:22.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:41:22.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:41:22.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[30017]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:22.639 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:22.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[30017]: osdmap e44: 7 total, 7 up, 7 in 2024-09-17T11:41:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:41:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[40223]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:22 smithi049 ceph-mon[40223]: osdmap e44: 7 total, 7 up, 7 in 2024-09-17T11:41:22.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:22 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:41:22.913 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:22 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:41:22.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:22 smithi129 ceph-mon[33364]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:22.914 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:22 smithi129 ceph-mon[33364]: osdmap e44: 7 total, 7 up, 7 in 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 -- 172.21.15.129:0/2754370439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c103b60 msgr2=0x7f3b4c103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/2754370439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c103b60 0x7f3b4c103fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009930 tx=0x7f3b3802a420 comp rx=0 tx=0).stop 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 -- 172.21.15.129:0/2754370439 shutdown_connections 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/2754370439 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c104520 0x7f3b4c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/2754370439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c103b60 0x7f3b4c103fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/2754370439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c102a00 0x7f3b4c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:23.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 -- 172.21.15.129:0/2754370439 >> 172.21.15.129:0/2754370439 conn(0x7f3b4c0fe2f0 msgr2=0x7f3b4c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 -- 172.21.15.129:0/2754370439 shutdown_connections 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.136+0000 7f3b52ca1640 1 -- 172.21.15.129:0/2754370439 wait complete. 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 Processor -- start 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 -- start start 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 0x7f3b4c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c103b60 0x7f3b4c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10ddb0 con 0x7f3b4c103b60 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10dc30 con 0x7f3b4c104520 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b52ca1640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c10df30 con 0x7f3b4c102a00 2024-09-17T11:41:23.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35246/0 (socket says 172.21.15.129:35246) 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 -- 172.21.15.129:0/964256443 learned_addr learned my addr 172.21.15.129:0/964256443 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b4bfff640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c103b60 0x7f3b4c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b50a3d640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 0x7f3b4c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 -- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 msgr2=0x7f3b4c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 0x7f3b4c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 -- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c103b60 msgr2=0x7f3b4c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c103b60 0x7f3b4c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b5123e640 1 -- 172.21.15.129:0/964256443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c19a0a0 con 0x7f3b4c104520 2024-09-17T11:41:23.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.137+0000 7f3b50a3d640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 0x7f3b4c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b5123e640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b970 tx=0x7f3b4400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44019070 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3b44014440 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c19a390 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440075d0 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.138+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a0a70 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.139+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c102e00 con 0x7f3b4c104520 2024-09-17T11:41:23.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.139+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f3b4401c070 con 0x7f3b4c104520 2024-09-17T11:41:23.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.140+0000 7f3b49ffb640 1 --2- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 0x7f3b3c07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:23.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.140+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 3789+0+0 (secure 0 0 0) 0x7f3b4409ca70 con 0x7f3b4c104520 2024-09-17T11:41:23.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.140+0000 7f3b50a3d640 1 --2- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 0x7f3b3c07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:23.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.141+0000 7f3b50a3d640 1 --2- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 0x7f3b3c07d1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b400058c0 tx=0x7f3b40005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:23.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.144+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3b44068b50 con 0x7f3b4c104520 2024-09-17T11:41:23.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:23.334+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3b4c02e780 con 0x7f3b3c07ace0 2024-09-17T11:41:23.847 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:23 smithi129 ceph-mon[33364]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:23.847 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:23 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:41:23.847 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:23 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:41:23.847 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:23 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:23.847 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:23 smithi129 ceph-mon[33364]: osdmap e45: 7 total, 7 up, 7 in 2024-09-17T11:41:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[40223]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:41:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:41:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[40223]: osdmap e45: 7 total, 7 up, 7 in 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[30017]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:23 smithi049 ceph-mon[30017]: osdmap e45: 7 total, 7 up, 7 in 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi129 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.141 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.142 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:24 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi129 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi129 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:25.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:24 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/1187277613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"}]': finished 2024-09-17T11:41:27.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: osdmap e46: 8 total, 7 up, 8 in 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/732350173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/1187277613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"}]': finished 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: osdmap e46: 8 total, 7 up, 8 in 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:27.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/732350173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:27.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/1187277613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"} : dispatch 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3"}]': finished 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: osdmap e46: 8 total, 7 up, 8 in 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:27.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/732350173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-17T11:41:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:28 smithi049 ceph-mon[30017]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:28 smithi049 ceph-mon[40223]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:28 smithi129 ceph-mon[33364]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:31.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:30 smithi049 ceph-mon[30017]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:30 smithi049 ceph-mon[40223]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:31.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:30 smithi129 ceph-mon[33364]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:32 smithi049 ceph-mon[30017]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:33.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:32 smithi049 ceph-mon[40223]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:32 smithi129 ceph-mon[33364]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:34.037 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-17T11:41:34.037 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:33 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:34.037 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:33 smithi129 ceph-mon[33364]: Deploying daemon osd.7 on smithi129 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[30017]: Deploying daemon osd.7 on smithi129 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:33 smithi049 ceph-mon[40223]: Deploying daemon osd.7 on smithi129 2024-09-17T11:41:35.127 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:34 smithi129 ceph-mon[33364]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:34 smithi049 ceph-mon[30017]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:34 smithi049 ceph-mon[40223]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:37.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:36 smithi129 ceph-mon[33364]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:37.705 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:36 smithi049 ceph-mon[30017]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:37.706 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:36 smithi049 ceph-mon[40223]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:38 smithi129 ceph-mon[33364]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:38 smithi049 ceph-mon[30017]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:38 smithi049 ceph-mon[40223]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:39 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:39 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:39 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:39 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:41 smithi129 ceph-mon[33364]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:41.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:41 smithi049 ceph-mon[30017]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:41 smithi049 ceph-mon[40223]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:42.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.637+0000 7f3b49ffb640 1 -- 172.21.15.129:0/964256443 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b4c02e780 con 0x7f3b3c07ace0 2024-09-17T11:41:42.640 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 7 on host 'smithi129' 2024-09-17T11:41:42.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 msgr2=0x7f3b3c07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:42.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 0x7f3b3c07d1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b400058c0 tx=0x7f3b40005ec0 comp rx=0 tx=0).stop 2024-09-17T11:41:42.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 msgr2=0x7f3b4c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:42.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b970 tx=0x7f3b4400be30 comp rx=0 tx=0).stop 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 shutdown_connections 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3b3c07ace0 0x7f3b3c07d1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b4c104520 0x7f3b4c199800 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b4c103b60 0x7f3b4c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 --2- 172.21.15.129:0/964256443 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3b4c102a00 0x7f3b4c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.639+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 >> 172.21.15.129:0/964256443 conn(0x7f3b4c0fe2f0 msgr2=0x7f3b4c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.640+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 shutdown_connections 2024-09-17T11:41:42.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:42.640+0000 7f3b52ca1640 1 -- 172.21.15.129:0/964256443 wait complete. 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:42 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.277 DEBUG:teuthology.orchestra.run.smithi129:osd.7> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.7.service 2024-09-17T11:41:43.280 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-17T11:41:43.280 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T11:41:43.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:43.311 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.312 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:42 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:43.551 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:41:44.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- 172.21.15.49:0/4243058322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a41021f0 msgr2=0x7f74a4102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:44.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 --2- 172.21.15.49:0/4243058322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a41021f0 0x7f74a4102670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7494009900 tx=0x7f749402a410 comp rx=0 tx=0).stop 2024-09-17T11:41:44.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- 172.21.15.49:0/4243058322 shutdown_connections 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 --2- 172.21.15.49:0/4243058322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a4102bb0 0x7f74a4109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 --2- 172.21.15.49:0/4243058322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a41021f0 0x7f74a4102670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 --2- 172.21.15.49:0/4243058322 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4101090 0x7f74a4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- 172.21.15.49:0/4243058322 >> 172.21.15.49:0/4243058322 conn(0x7f74a40fc960 msgr2=0x7f74a40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- 172.21.15.49:0/4243058322 shutdown_connections 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- 172.21.15.49:0/4243058322 wait complete. 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 Processor -- start 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.998+0000 7f74abba5640 1 -- start start 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:44.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 0x7f74a4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 0x7f74a4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410dda0 con 0x7f74a4101090 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410dc20 con 0x7f74a41021f0 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74abba5640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f74a410df20 con 0x7f74a4102bb0 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74aa142640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 0x7f74a4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39554/0 (socket says 172.21.15.49:39554) 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 -- 172.21.15.49:0/1138627501 learned_addr learned my addr 172.21.15.49:0/1138627501 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9140640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 0x7f74a4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 -- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 msgr2=0x7f74a4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 0x7f74a4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 -- 172.21.15.49:0/1138627501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 msgr2=0x7f74a4198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 0x7f74a4198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 -- 172.21.15.49:0/1138627501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a419a000 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74aa142640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 0x7f74a4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9140640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 0x7f74a4198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:43.999+0000 7f74a9941640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74a000ea60 tx=0x7f74a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.000+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000cde0 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.000+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f74a00035f0 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.000+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00044a0 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.000+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a419a2f0 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.000+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a410aed0 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.001+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a4101490 con 0x7f74a4101090 2024-09-17T11:41:44.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.001+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f74a0004000 con 0x7f74a4101090 2024-09-17T11:41:44.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.002+0000 7f749affd640 1 --2- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 0x7f749007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:44.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.002+0000 7f74a9140640 1 --2- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 0x7f749007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:44.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.002+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 3904+0+0 (secure 0 0 0) 0x7f74a009d810 con 0x7f74a4101090 2024-09-17T11:41:44.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.003+0000 7f74a9140640 1 --2- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 0x7f749007d1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7494005750 tx=0x7f7494005e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:44.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.007+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f74a0069900 con 0x7f74a4101090 2024-09-17T11:41:44.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.196+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f74a402e5c0 con 0x7f74a4101090 2024-09-17T11:41:44.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.197+0000 7f749affd640 1 -- 172.21.15.49:0/1138627501 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v46) ==== 74+0+130 (secure 0 0 0) 0x7f74a006e7b0 con 0x7f74a4101090 2024-09-17T11:41:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 msgr2=0x7f749007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 0x7f749007d1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7494005750 tx=0x7f7494005e00 comp rx=0 tx=0).stop 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 msgr2=0x7f74a41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74a000ea60 tx=0x7f74a000ef20 comp rx=0 tx=0).stop 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 shutdown_connections 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> v2:172.21.15.49:6800/1112797402 conn(0x7f749007ace0 0x7f749007d1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f74a4102bb0 0x7f74a4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74a41021f0 0x7f74a4198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 --2- 172.21.15.49:0/1138627501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74a4101090 0x7f74a41984d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 >> 172.21.15.49:0/1138627501 conn(0x7f74a40fc960 msgr2=0x7f74a40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 shutdown_connections 2024-09-17T11:41:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:44.199+0000 7f74abba5640 1 -- 172.21.15.49:0/1138627501 wait complete. 2024-09-17T11:41:44.508 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 11:41:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T11:41:44.403+0000 7f53be3d4640 -1 osd.7 0 log_to_monitors true 2024-09-17T11:41:45.015 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":46,"num_osds":8,"num_up_osds":7,"osd_up_since":1726573281,"num_in_osds":8,"osd_in_since":1726573285,"num_remapped_pgs":0} 2024-09-17T11:41:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:44 smithi129 ceph-mon[33364]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1138627501' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:44 smithi129 ceph-mon[33364]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:44 smithi129 ceph-mon[33364]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[30017]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1138627501' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[30017]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[30017]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[40223]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:45.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1138627501' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:45.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[40223]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:45.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:44 smithi049 ceph-mon[40223]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-17T11:41:46.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: osdmap e47: 8 total, 7 up, 8 in 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.258 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:46.294 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: osdmap e47: 8 total, 7 up, 8 in 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.313 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[30017]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: from='osd.7 v2:172.21.15.129:6812/1293620562' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: osdmap e47: 8 total, 7 up, 8 in 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]} : dispatch 2024-09-17T11:41:46.314 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:45 smithi049 ceph-mon[40223]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-17T11:41:46.676 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T11:41:46.393+0000 7f53bab6b640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T11:41:46.676 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T11:41:46.399+0000 7f53b2176640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T11:41:46.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 -- 172.21.15.49:0/2107480955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9404108700 msgr2=0x7f940410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 --2- 172.21.15.49:0/2107480955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9404108700 0x7f940410abc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009ca0 tx=0x7f93fc02a8c0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 -- 172.21.15.49:0/2107480955 shutdown_connections 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 --2- 172.21.15.49:0/2107480955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9404108700 0x7f940410abc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 --2- 172.21.15.49:0/2107480955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404105b50 0x7f9404107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 --2- 172.21.15.49:0/2107480955 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f940406b6b0 0x7f94041055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 -- 172.21.15.49:0/2107480955 >> 172.21.15.49:0/2107480955 conn(0x7f94040fd1b0 msgr2=0x7f94040ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 -- 172.21.15.49:0/2107480955 shutdown_connections 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.862+0000 7f940b67e640 1 -- 172.21.15.49:0/2107480955 wait complete. 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 Processor -- start 2024-09-17T11:41:46.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 -- start start 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 0x7f9404102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 0x7f9404101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404108700 0x7f9404101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f940410dd10 con 0x7f940406b6b0 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940410db90 con 0x7f9404108700 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940b67e640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f940410de90 con 0x7f9404105b50 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f940941a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 0x7f9404102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f9408c19640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 0x7f9404101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f9408c19640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 0x7f9404101560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:38068/0 (socket says 172.21.15.49:38068) 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.863+0000 7f9408c19640 1 -- 172.21.15.49:0/2797315548 learned_addr learned my addr 172.21.15.49:0/2797315548 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f9409c1b640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404108700 0x7f9404101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 -- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 msgr2=0x7f9404101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:46.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 0x7f9404101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 -- 172.21.15.49:0/2797315548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404108700 msgr2=0x7f9404101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404108700 0x7f9404101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 -- 172.21.15.49:0/2797315548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9404102360 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940941a640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 0x7f9404102cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f940000e9f0 tx=0x7f940000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940000cd80 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f94000035f0 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94041a2590 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000044a0 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.864+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041a2af0 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.865+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9404105fb0 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.867+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f9400004000 con 0x7f940406b6b0 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.867+0000 7f93fa7fc640 1 --2- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 0x7f93f007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.867+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 3941+0+0 (secure 0 0 0) 0x7f940009d850 con 0x7f940406b6b0 2024-09-17T11:41:46.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.867+0000 7f9408c19640 1 --2- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 0x7f93f007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:46.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.868+0000 7f9408c19640 1 --2- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 0x7f93f007d1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005bc0 tx=0x7f93f400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:46.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:46.872+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f9400065530 con 0x7f940406b6b0 2024-09-17T11:41:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.066+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9404063d50 con 0x7f940406b6b0 2024-09-17T11:41:47.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.066+0000 7f93fa7fc640 1 -- 172.21.15.49:0/2797315548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v49) ==== 74+0+130 (secure 0 0 0) 0x7f9400069910 con 0x7f940406b6b0 2024-09-17T11:41:47.069 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 msgr2=0x7f93f007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 0x7f93f007d1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005bc0 tx=0x7f93f400b480 comp rx=0 tx=0).stop 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 msgr2=0x7f9404102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 0x7f9404102cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f940000e9f0 tx=0x7f940000eeb0 comp rx=0 tx=0).stop 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 shutdown_connections 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> v2:172.21.15.49:6800/1112797402 conn(0x7f93f007ace0 0x7f93f007d1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404108700 0x7f9404101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:47.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f9404105b50 0x7f9404101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:47.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 --2- 172.21.15.49:0/2797315548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940406b6b0 0x7f9404102cf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:47.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 >> 172.21.15.49:0/2797315548 conn(0x7f94040fd1b0 msgr2=0x7f9404103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:47.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 shutdown_connections 2024-09-17T11:41:47.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:47.070+0000 7f940b67e640 1 -- 172.21.15.49:0/2797315548 wait complete. 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: osdmap e48: 8 total, 7 up, 8 in 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: Detected new or changed devices on smithi129 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.343 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='osd.7 ' entity='osd.7' 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: osd.7 v2:172.21.15.129:6812/1293620562 boot 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: osdmap e49: 8 total, 8 up, 8 in 2024-09-17T11:41:47.344 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: osdmap e48: 8 total, 7 up, 8 in 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: Detected new or changed devices on smithi129 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='osd.7 ' entity='osd.7' 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: osd.7 v2:172.21.15.129:6812/1293620562 boot 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: osdmap e49: 8 total, 8 up, 8 in 2024-09-17T11:41:47.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:46 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-17T11:41:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: osdmap e48: 8 total, 7 up, 8 in 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: Detected new or changed devices on smithi129 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='osd.7 ' entity='osd.7' 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: osd.7 v2:172.21.15.129:6812/1293620562 boot 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: osdmap e49: 8 total, 8 up, 8 in 2024-09-17T11:41:47.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:46 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:41:47.776 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1726573306,"num_in_osds":8,"osd_in_since":1726573285,"num_remapped_pgs":1} 2024-09-17T11:41:47.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T11:41:48.034 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:41:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[30017]: purged_snaps scrub starts 2024-09-17T11:41:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[30017]: purged_snaps scrub ok 2024-09-17T11:41:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2797315548' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[30017]: osdmap e50: 8 total, 8 up, 8 in 2024-09-17T11:41:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[40223]: purged_snaps scrub starts 2024-09-17T11:41:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[40223]: purged_snaps scrub ok 2024-09-17T11:41:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2797315548' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:47 smithi049 ceph-mon[40223]: osdmap e50: 8 total, 8 up, 8 in 2024-09-17T11:41:48.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:47 smithi129 ceph-mon[33364]: purged_snaps scrub starts 2024-09-17T11:41:48.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:47 smithi129 ceph-mon[33364]: purged_snaps scrub ok 2024-09-17T11:41:48.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2797315548' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-17T11:41:48.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:47 smithi129 ceph-mon[33364]: osdmap e50: 8 total, 8 up, 8 in 2024-09-17T11:41:48.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- 172.21.15.49:0/963992696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c104590 msgr2=0x7f003c10ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 --2- 172.21.15.49:0/963992696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c104590 0x7f003c10ca80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f002c009900 tx=0x7f002c02a410 comp rx=0 tx=0).stop 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- 172.21.15.49:0/963992696 shutdown_connections 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 --2- 172.21.15.49:0/963992696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c10d0d0 0x7f003c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 --2- 172.21.15.49:0/963992696 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c104590 0x7f003c10ca80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 --2- 172.21.15.49:0/963992696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c103c70 0x7f003c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- 172.21.15.49:0/963992696 >> 172.21.15.49:0/963992696 conn(0x7f003c0fd6d0 msgr2=0x7f003c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- 172.21.15.49:0/963992696 shutdown_connections 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- 172.21.15.49:0/963992696 wait complete. 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 Processor -- start 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.459+0000 7f004387c640 1 -- start start 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c104590 0x7f003c10bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1124a0 con 0x7f003c103c70 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112320 con 0x7f003c104590 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f004387c640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f003c112620 con 0x7f003c10d0d0 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041618640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041618640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39582/0 (socket says 172.21.15.49:39582) 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:38076/0 (socket says 172.21.15.49:38076) 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041618640 1 -- 172.21.15.49:0/2980541416 learned_addr learned my addr 172.21.15.49:0/2980541416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0040e17640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c104590 0x7f003c10bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 -- 172.21.15.49:0/2980541416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c104590 msgr2=0x7f003c10bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c104590 0x7f003c10bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 -- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 msgr2=0x7f003c10b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 -- 172.21.15.49:0/2980541416 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c1a3f40 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041618640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f0041e19640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f003400ec50 tx=0x7f003400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.460+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034003b70 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.461+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c1a4230 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.461+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f0034003d10 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.461+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034010650 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.461+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1aba20 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.461+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c104050 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.462+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f0034004020 con 0x7f003c10d0d0 2024-09-17T11:41:48.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.463+0000 7f00327fc640 1 --2- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 0x7f002807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.463+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 4081+0+0 (secure 0 0 0) 0x7f003409dbd0 con 0x7f003c10d0d0 2024-09-17T11:41:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.463+0000 7f0041618640 1 --2- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 0x7f002807d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.463+0000 7f0041618640 1 --2- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 0x7f002807d1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00380104e0 tx=0x7f003800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:48.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.468+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f00340657a0 con 0x7f003c10d0d0 2024-09-17T11:41:48.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.654+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f003c1abc50 con 0x7f003c10d0d0 2024-09-17T11:41:48.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.655+0000 7f00327fc640 1 -- 172.21.15.49:0/2980541416 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+11519 (secure 0 0 0) 0x7f0034069b80 con 0x7f003c10d0d0 2024-09-17T11:41:48.657 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:41:48.657 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","created":"2024-09-17T11:36:43.058933+0000","modified":"2024-09-17T11:41:47.962227+0000","last_up_change":"2024-09-17T11:41:46.960860+0000","last_in_change":"2024-09-17T11:41:25.982951+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T11:39:42.144376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a304fc7-f010-473e-ad18-e04e9b886490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":501081646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":501081646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":501081646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6803","nonce":501081646}]},"public_addr":"172.21.15.49:6801/501081646","cluster_addr":"172.21.15.49:6802/501081646","heartbeat_back_addr":"172.21.15.49:6804/501081646","heartbeat_front_addr":"172.21.15.49:6803/501081646","state":["exists","up"]},{"osd":1,"uuid":"b1d4bec5-5437-46c5-b724-270c2867bd33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":1599533781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1599533781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1599533781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6807","nonce":1599533781}]},"public_addr":"172.21.15.49:6805/1599533781","cluster_addr":"172.21.15.49:6806/1599533781","heartbeat_back_addr":"172.21.15.49:6808/1599533781","heartbeat_front_addr":"172.21.15.49:6807/1599533781","state":["exists","up"]},{"osd":2,"uuid":"ef100460-beb3-49ed-b03c-764b2bd5c333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":185342269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":185342269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":185342269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6811","nonce":185342269}]},"public_addr":"172.21.15.49:6809/185342269","cluster_addr":"172.21.15.49:6810/185342269","heartbeat_back_addr":"172.21.15.49:6812/185342269","heartbeat_front_addr":"172.21.15.49:6811/185342269","state":["exists","up"]},{"osd":3,"uuid":"b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":1277715496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1277715496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1277715496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":1277715496}]},"public_addr":"172.21.15.49:6813/1277715496","cluster_addr":"172.21.15.49:6814/1277715496","heartbeat_back_addr":"172.21.15.49:6816/1277715496","heartbeat_front_addr":"172.21.15.49:6815/1277715496","state":["exists","up"]},{"osd":4,"uuid":"9500ec09-085d-45ae-8356-ea20c1a7d064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2156637849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":2156637849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2156637849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2156637849}]},"public_addr":"172.21.15.129:6800/2156637849","cluster_addr":"172.21.15.129:6801/2156637849","heartbeat_back_addr":"172.21.15.129:6803/2156637849","heartbeat_front_addr":"172.21.15.129:6802/2156637849","state":["exists","up"]},{"osd":5,"uuid":"a9691398-5234-4fb1-ba46-1369ac2b43cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2449624152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":2449624152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2449624152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2449624152}]},"public_addr":"172.21.15.129:6804/2449624152","cluster_addr":"172.21.15.129:6805/2449624152","heartbeat_back_addr":"172.21.15.129:6807/2449624152","heartbeat_front_addr":"172.21.15.129:6806/2449624152","state":["exists","up"]},{"osd":6,"uuid":"34249b34-b42f-4289-9797-11f3e971069e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3044459329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6809","nonce":3044459329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3044459329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3044459329}]},"public_addr":"172.21.15.129:6808/3044459329","cluster_addr":"172.21.15.129:6809/3044459329","heartbeat_back_addr":"172.21.15.129:6811/3044459329","heartbeat_front_addr":"172.21.15.129:6810/3044459329","state":["exists","up"]},{"osd":7,"uuid":"9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1293620562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6813","nonce":1293620562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1293620562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1293620562}]},"public_addr":"172.21.15.129:6812/1293620562","cluster_addr":"172.21.15.129:6813/1293620562","heartbeat_back_addr":"172.21.15.129:6815/1293620562","heartbeat_front_addr":"172.21.15.129:6814/1293620562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:38:47.278603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:13.199579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:39.193605+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:04.375752+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:27.617287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:54.268575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:20.430993+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:45.446088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6800/1315254490":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/2860709752":"2024-09-18T11:37:06.723518+0000","172.21.15.49:6800/1073451895":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3769210869":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/708267795":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3594659540":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/546102581":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/3939921930":"2024-09-18T11:37:06.723518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.657+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 msgr2=0x7f002807d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.657+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 0x7f002807d1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00380104e0 tx=0x7f003800b410 comp rx=0 tx=0).stop 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.657+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 msgr2=0x7f003c1a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.657+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f003400ec50 tx=0x7f003400c6a0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 shutdown_connections 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> v2:172.21.15.49:6800/1112797402 conn(0x7f002807ace0 0x7f002807d1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f003c10d0d0 0x7f003c1a3770 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f003c104590 0x7f003c10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 --2- 172.21.15.49:0/2980541416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f003c103c70 0x7f003c10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:48.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 >> 172.21.15.49:0/2980541416 conn(0x7f003c0fd6d0 msgr2=0x7f003c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:48.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 shutdown_connections 2024-09-17T11:41:48.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:48.658+0000 7f004387c640 1 -- 172.21.15.49:0/2980541416 wait complete. 2024-09-17T11:41:49.323 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[40223]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:49.324 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2980541416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:41:49.324 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[40223]: osdmap e51: 8 total, 8 up, 8 in 2024-09-17T11:41:49.324 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[30017]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:49.324 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2980541416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:41:49.324 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:48 smithi049 ceph-mon[30017]: osdmap e51: 8 total, 8 up, 8 in 2024-09-17T11:41:49.354 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T11:39:42.144376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T11:41:49.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T11:41:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:48 smithi129 ceph-mon[33364]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2980541416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:41:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:48 smithi129 ceph-mon[33364]: osdmap e51: 8 total, 8 up, 8 in 2024-09-17T11:41:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:41:50.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1014886056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 msgr2=0x7f3f8c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:50.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1014886056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c103180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009860 tx=0x7f3f7c02a340 comp rx=0 tx=0).stop 2024-09-17T11:41:50.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1014886056 shutdown_connections 2024-09-17T11:41:50.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1014886056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c109130 0x7f3f8c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1014886056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c1087b0 0x7f3f8c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1014886056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c103180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1014886056 >> 172.21.15.49:0/1014886056 conn(0x7f3f8c0fe810 msgr2=0x7f3f8c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1014886056 shutdown_connections 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.038+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1014886056 wait complete. 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 Processor -- start 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 -- start start 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c1087b0 0x7f3f8c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 0x7f3f8c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c112500 con 0x7f3f8c109130 2024-09-17T11:41:50.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c112380 con 0x7f3f8c1087b0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f91f4a640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c112680 con 0x7f3f8c102da0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.039+0000 7f3f8b7fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:38080/0 (socket says 172.21.15.49:38080) 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 -- 172.21.15.49:0/1177566079 learned_addr learned my addr 172.21.15.49:0/1177566079 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8bfff640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 0x7f3f8c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8affd640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c1087b0 0x7f3f8c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 -- 172.21.15.49:0/1177566079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c1087b0 msgr2=0x7f3f8c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c1087b0 0x7f3f8c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 -- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 msgr2=0x7f3f8c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 0x7f3f8c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 -- 172.21.15.49:0/1177566079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c19ea40 con 0x7f3f8c102da0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8bfff640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 0x7f3f8c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f8b7fe640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009980 tx=0x7f3f7c033d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:50.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c03f020 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c19ecd0 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3f7c0044c0 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c037040 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.040+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1a53f0 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.041+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c103180 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.042+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f3f7c042020 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.042+0000 7f3f88ff9640 1 --2- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 0x7f3f7407d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.043+0000 7f3f8affd640 1 --2- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 0x7f3f7407d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.043+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 4081+0+0 (secure 0 0 0) 0x7f3f7c0bd570 con 0x7f3f8c102da0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.043+0000 7f3f8affd640 1 --2- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 0x7f3f7407d1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c102220 tx=0x7f3f80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:50.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.047+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3f7c049020 con 0x7f3f8c102da0 2024-09-17T11:41:50.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.231+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f8c10af10 con 0x7f3f8c102da0 2024-09-17T11:41:50.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.231+0000 7f3f88ff9640 1 -- 172.21.15.49:0/1177566079 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f3f7c089520 con 0x7f3f8c102da0 2024-09-17T11:41:50.233 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-09-17T11:41:50.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 msgr2=0x7f3f7407d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:50.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 0x7f3f7407d1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c102220 tx=0x7f3f80009260 comp rx=0 tx=0).stop 2024-09-17T11:41:50.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 msgr2=0x7f3f8c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009980 tx=0x7f3f7c033d30 comp rx=0 tx=0).stop 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 shutdown_connections 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f7407ace0 0x7f3f7407d1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c109130 0x7f3f8c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f8c1087b0 0x7f3f8c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 --2- 172.21.15.49:0/1177566079 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f8c102da0 0x7f3f8c198990 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 >> 172.21.15.49:0/1177566079 conn(0x7f3f8c0fe810 msgr2=0x7f3f8c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 shutdown_connections 2024-09-17T11:41:50.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:41:50.233+0000 7f3f91f4a640 1 -- 172.21.15.49:0/1177566079 wait complete. 2024-09-17T11:41:51.040 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi049 2024-09-17T11:41:51.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi049=foo.a' 2024-09-17T11:41:51.299 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:51.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:51 smithi049 ceph-mon[40223]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:51.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1177566079' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-17T11:41:51.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:51 smithi049 ceph-mon[30017]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:51.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1177566079' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-17T11:41:51.420 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:51 smithi129 ceph-mon[33364]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:51.420 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1177566079' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.760+0000 7f7170416640 1 -- 172.21.15.129:0/2711010533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168108730 msgr2=0x7f716810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.760+0000 7f7170416640 1 --2- 172.21.15.129:0/2711010533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168108730 0x7f716810abf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7160009ca0 tx=0x7f716002a8c0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- 172.21.15.129:0/2711010533 shutdown_connections 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- 172.21.15.129:0/2711010533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168108730 0x7f716810abf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- 172.21.15.129:0/2711010533 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7168105ba0 0x7f7168107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- 172.21.15.129:0/2711010533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f716806c030 0x7f7168105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- 172.21.15.129:0/2711010533 >> 172.21.15.129:0/2711010533 conn(0x7f71680fd1d0 msgr2=0x7f71680ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- 172.21.15.129:0/2711010533 shutdown_connections 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- 172.21.15.129:0/2711010533 wait complete. 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 Processor -- start 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- start start 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 0x7f7168198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 0x7f71681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f716810de10 con 0x7f7168108730 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f716810dc90 con 0x7f7168105ba0 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f7170416640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f716810df90 con 0x7f716806c030 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f716d9b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f716d9b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46632/0 (socket says 172.21.15.129:46632) 2024-09-17T11:41:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f716d9b1640 1 -- 172.21.15.129:0/1611435471 learned_addr learned my addr 172.21.15.129:0/1611435471 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.761+0000 7f716e1b2640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 0x7f7168198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716e9b3640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 0x7f71681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 -- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 msgr2=0x7f7168198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 0x7f7168198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 -- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 msgr2=0x7f71681998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 0x7f71681998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 -- 172.21.15.129:0/1611435471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716819a170 con 0x7f7168105ba0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716e1b2640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 0x7f7168198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716e9b3640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 0x7f71681998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f716d9b1640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f715800b4a0 tx=0x7f715800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158014070 con 0x7f7168105ba0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f7158010a90 con 0x7f7168105ba0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716819a460 con 0x7f7168105ba0 2024-09-17T11:41:51.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158017b80 con 0x7f7168105ba0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.762+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a0b40 con 0x7f7168105ba0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.763+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7168106060 con 0x7f7168105ba0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.764+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f7158017070 con 0x7f7168105ba0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.764+0000 7f715f7fe640 1 --2- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 0x7f715407d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.764+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4081+0+0 (secure 0 0 0) 0x7f715809cc60 con 0x7f7168105ba0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.764+0000 7f716e1b2640 1 --2- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 0x7f715407d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.765+0000 7f716e1b2640 1 --2- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 0x7f715407d1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f7164008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:51.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.769+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f71580648b0 con 0x7f7168105ba0 2024-09-17T11:41:51.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.954+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi049=foo.a", "target": ["mon-mgr", ""]}) -- 0x7f7168104e80 con 0x7f715407ace0 2024-09-17T11:41:51.964 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled rgw.foo.a update... 2024-09-17T11:41:51.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.962+0000 7f715f7fe640 1 -- 172.21.15.129:0/1611435471 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f7168104e80 con 0x7f715407ace0 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.964+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 msgr2=0x7f715407d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.964+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 0x7f715407d1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f7164008040 comp rx=0 tx=0).stop 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 msgr2=0x7f7168198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f715800b4a0 tx=0x7f715800b960 comp rx=0 tx=0).stop 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 shutdown_connections 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> v2:172.21.15.49:6800/1112797402 conn(0x7f715407ace0 0x7f715407d1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7168108730 0x7f71681998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7168105ba0 0x7f7168198c90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 --2- 172.21.15.129:0/1611435471 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f716806c030 0x7f7168198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:51.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 >> 172.21.15.129:0/1611435471 conn(0x7f71680fd1d0 msgr2=0x7f7168107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:51.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 shutdown_connections 2024-09-17T11:41:51.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:51.965+0000 7f7170416640 1 -- 172.21.15.129:0/1611435471 wait complete. 2024-09-17T11:41:52.642 DEBUG:teuthology.orchestra.run.smithi049:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T11:41:52.645 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi129 2024-09-17T11:41:52.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-17T11:41:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi049=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:52 smithi129 ceph-mon[33364]: Deploying daemon rgw.foo.a on smithi049 2024-09-17T11:41:53.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 -- 172.21.15.129:0/2156211330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c104520 msgr2=0x7f3f5c10aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:53.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 --2- 172.21.15.129:0/2156211330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c104520 0x7f3f5c10aea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009cd0 tx=0x7f3f5402a8e0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 -- 172.21.15.129:0/2156211330 shutdown_connections 2024-09-17T11:41:53.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 --2- 172.21.15.129:0/2156211330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c104520 0x7f3f5c10aea0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 --2- 172.21.15.129:0/2156211330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c103b60 0x7f3f5c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 --2- 172.21.15.129:0/2156211330 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c102a00 0x7f3f5c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 -- 172.21.15.129:0/2156211330 >> 172.21.15.129:0/2156211330 conn(0x7f3f5c0fe2f0 msgr2=0x7f3f5c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 -- 172.21.15.129:0/2156211330 shutdown_connections 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.333+0000 7f3f64130640 1 -- 172.21.15.129:0/2156211330 wait complete. 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 Processor -- start 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 -- start start 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c103b60 0x7f3f5c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c104520 0x7f3f5c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10dd50 con 0x7f3f5c104520 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10dbd0 con 0x7f3f5c102a00 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f64130640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c10ded0 con 0x7f3f5c103b60 2024-09-17T11:41:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47046/0 (socket says 172.21.15.129:47046) 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 -- 172.21.15.129:0/3035623989 learned_addr learned my addr 172.21.15.129:0/3035623989 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f616cb640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c103b60 0x7f3f5c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f626cd640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c104520 0x7f3f5c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 -- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c103b60 msgr2=0x7f3f5c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c103b60 0x7f3f5c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 -- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c104520 msgr2=0x7f3f5c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c104520 0x7f3f5c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.334+0000 7f3f61ecc640 1 -- 172.21.15.129:0/3035623989 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c199fd0 con 0x7f3f5c102a00 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f61ecc640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800d770 tx=0x7f3f5800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58017070 con 0x7f3f5c102a00 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3f580054a0 con 0x7f3f5c102a00 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c19a2c0 con 0x7f3f5c102a00 2024-09-17T11:41:53.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004c40 con 0x7f3f5c102a00 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.335+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1a09a0 con 0x7f3f5c102a00 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.336+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c102e00 con 0x7f3f5c102a00 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.336+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f3f58004de0 con 0x7f3f5c102a00 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.337+0000 7f3f52ffd640 1 --2- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 0x7f3f4807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.337+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4081+0+0 (secure 0 0 0) 0x7f3f5809d0b0 con 0x7f3f5c102a00 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.337+0000 7f3f616cb640 1 --2- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 0x7f3f4807d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.338+0000 7f3f616cb640 1 --2- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 0x7f3f4807d1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00bbe0 tx=0x7f3f4c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:53.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.341+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3f58069060 con 0x7f3f5c102a00 2024-09-17T11:41:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi049=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:53.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[30017]: Deploying daemon rgw.foo.a on smithi049 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi049=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:41:53.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:53.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:52 smithi049 ceph-mon[40223]: Deploying daemon rgw.foo.a on smithi049 2024-09-17T11:41:53.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.517+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7f3f5c02e5c0 con 0x7f3f5c102a00 2024-09-17T11:41:53.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.970+0000 7f3f52ffd640 1 -- 172.21.15.129:0/3035623989 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v52) ==== 160+0+0 (secure 0 0 0) 0x7f3f5806df10 con 0x7f3f5c102a00 2024-09-17T11:41:53.972 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'datapool' created 2024-09-17T11:41:53.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 msgr2=0x7f3f4807d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:53.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 0x7f3f4807d1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00bbe0 tx=0x7f3f4c00b390 comp rx=0 tx=0).stop 2024-09-17T11:41:53.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 msgr2=0x7f3f5c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800d770 tx=0x7f3f5800dc30 comp rx=0 tx=0).stop 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 shutdown_connections 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3f4807ace0 0x7f3f4807d1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f5c104520 0x7f3f5c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3f5c103b60 0x7f3f5c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 --2- 172.21.15.129:0/3035623989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f5c102a00 0x7f3f5c1984a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 >> 172.21.15.129:0/3035623989 conn(0x7f3f5c0fe2f0 msgr2=0x7f3f5c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 shutdown_connections 2024-09-17T11:41:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:53.973+0000 7f3f64130640 1 -- 172.21.15.129:0/3035623989 wait complete. 2024-09-17T11:41:54.148 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/3035623989' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.148 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.148 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/3035623989' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.148 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.278 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/3035623989' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.278 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-17T11:41:54.400 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 11:41:54 smithi049 podman[66210]: 2024-09-17 11:41:54.357033774 +0000 UTC m=+0.324020989 container start c6a5a4b09e8600d34251426abe23359149da6c1110c120e4b494d83923f144f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, RELEASE=main-7498493, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T11:41:54.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-17T11:41:54.867 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:54.900 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 11:41:54 smithi049 bash[66210]: c6a5a4b09e8600d34251426abe23359149da6c1110c120e4b494d83923f144f3 2024-09-17T11:41:54.900 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 11:41:54 smithi049 systemd[1]: Started Ceph rgw.foo.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:41:55.236 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:55.236 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: osdmap e52: 8 total, 8 up, 8 in 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.237 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: osdmap e52: 8 total, 8 up, 8 in 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.238 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:54 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: osdmap e52: 8 total, 8 up, 8 in 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: Saving service rgw.foo.a spec with placement smithi049=foo.a;count:1 2024-09-17T11:41:55.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:55.250 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:54 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:41:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: osdmap e53: 8 total, 8 up, 8 in 2024-09-17T11:41:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-17T11:41:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/3516815556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[30017]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: osdmap e53: 8 total, 8 up, 8 in 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-17T11:41:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/3516815556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T11:41:56.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:55 smithi049 ceph-mon[40223]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T11:41:56.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: osdmap e53: 8 total, 8 up, 8 in 2024-09-17T11:41:56.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-17T11:41:56.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/3516815556' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1430635413' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T11:41:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:55 smithi129 ceph-mon[33364]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: pgmap v175: 36 pgs: 14 creating+peering, 20 unknown, 2 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: Checking dashboard <-> RGW credentials 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: pgmap v175: 36 pgs: 14 creating+peering, 20 unknown, 2 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: Checking dashboard <-> RGW credentials 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:56 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: pgmap v175: 36 pgs: 14 creating+peering, 20 unknown, 2 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:41:57.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: Checking dashboard <-> RGW credentials 2024-09-17T11:41:57.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:41:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T11:41:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:56 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-17T11:41:58.296 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:58 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:58.296 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.296 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:58 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.296 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:58 smithi129 ceph-mon[33364]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T11:41:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[40223]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T11:41:58.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:41:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T11:41:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:58 smithi049 ceph-mon[30017]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T11:41:58.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.129 --placement '1;smithi129=iscsi.a' 2024-09-17T11:41:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:41:59.289 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:59 smithi129 ceph-mon[33364]: pgmap v178: 68 pgs: 21 creating+peering, 29 unknown, 18 active+clean; 578 KiB data, 224 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-17T11:41:59.289 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:59 smithi129 ceph-mon[33364]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T11:41:59.289 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.289 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.289 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:41:59 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 -- 172.21.15.129:0/1782613377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 msgr2=0x7f8c64102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:59.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1782613377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64102e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009cd0 tx=0x7f8c5402a8e0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 -- 172.21.15.129:0/1782613377 shutdown_connections 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1782613377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c6410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1782613377 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1782613377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64102e00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.377+0000 7f8c68f27640 1 -- 172.21.15.129:0/1782613377 >> 172.21.15.129:0/1782613377 conn(0x7f8c640fe2f0 msgr2=0x7f8c64100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- 172.21.15.129:0/1782613377 shutdown_connections 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- 172.21.15.129:0/1782613377 wait complete. 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 Processor -- start 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- start start 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c64075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410de10 con 0x7f8c64104520 2024-09-17T11:41:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410dc90 con 0x7f8c64102a00 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.378+0000 7f8c68f27640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6410df90 con 0x7f8c64103b60 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47078/0 (socket says 172.21.15.129:47078) 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 -- 172.21.15.129:0/1595969916 learned_addr learned my addr 172.21.15.129:0/1595969916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c62d9d640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c64075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c61d9b640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 -- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 msgr2=0x7f8c64075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 -- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 msgr2=0x7f8c64075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c64075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 -- 172.21.15.129:0/1595969916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64076340 con 0x7f8c64102a00 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c62d9d640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c64075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:41:59.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c61d9b640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c6259c640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097b0 tx=0x7f8c54003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c540044d0 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f8c54004670 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a2700 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54046750 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.379+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a2c90 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.380+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c64102e80 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.381+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f8c54030ab0 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.381+0000 7f8c5b7fe640 1 --2- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 0x7f8c5007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.381+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5549+0+0 (secure 0 0 0) 0x7f8c5402e030 con 0x7f8c64102a00 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.382+0000 7f8c61d9b640 1 --2- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 0x7f8c5007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:41:59.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.382+0000 7f8c61d9b640 1 --2- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 0x7f8c5007d1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009930 tx=0x7f8c4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:41:59.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.386+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f8c5404b060 con 0x7f8c64102a00 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[30017]: pgmap v178: 68 pgs: 21 creating+peering, 29 unknown, 18 active+clean; 578 KiB data, 224 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[30017]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[40223]: pgmap v178: 68 pgs: 21 creating+peering, 29 unknown, 18 active+clean; 578 KiB data, 224 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-17T11:41:59.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[40223]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T11:41:59.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:41:59 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-17T11:41:59.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.575+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.129", "placement": "1;smithi129=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f8c64063590 con 0x7f8c5007ace0 2024-09-17T11:41:59.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.585+0000 7f8c5b7fe640 1 -- 172.21.15.129:0/1595969916 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7f8c64063590 con 0x7f8c5007ace0 2024-09-17T11:41:59.586 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled iscsi.datapool update... 2024-09-17T11:41:59.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 msgr2=0x7f8c5007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:59.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 0x7f8c5007d1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009930 tx=0x7f8c4c008040 comp rx=0 tx=0).stop 2024-09-17T11:41:59.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 msgr2=0x7f8c64076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:41:59.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097b0 tx=0x7f8c54003e60 comp rx=0 tx=0).stop 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 shutdown_connections 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> v2:172.21.15.49:6800/1112797402 conn(0x7f8c5007ace0 0x7f8c5007d1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c64104520 0x7f8c64075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8c64103b60 0x7f8c64075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 --2- 172.21.15.129:0/1595969916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c64102a00 0x7f8c64076cd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:41:59.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 >> 172.21.15.129:0/1595969916 conn(0x7f8c640fe2f0 msgr2=0x7f8c640ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:41:59.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 shutdown_connections 2024-09-17T11:41:59.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:41:59.587+0000 7f8c68f27640 1 -- 172.21.15.129:0/1595969916 wait complete. 2024-09-17T11:42:00.195 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-17T11:42:00.195 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:42:00.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T11:42:00.226 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:42:00.227 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T11:42:00.252 DEBUG:teuthology.orchestra.run.smithi129:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-17T11:42:00.294 INFO:tasks.cephadm:Adding prometheus.a on smithi129 2024-09-17T11:42:00.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi129=a' 2024-09-17T11:42:00.576 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.129", "placement": "1;smithi129=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: Saving service iscsi.datapool spec with placement smithi129=iscsi.a;count:1 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: pgmap v181: 100 pgs: 21 creating+peering, 48 unknown, 31 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[40223]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.129", "placement": "1;smithi129=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: Saving service iscsi.datapool spec with placement smithi129=iscsi.a;count:1 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: pgmap v181: 100 pgs: 21 creating+peering, 48 unknown, 31 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:00 smithi049 ceph-mon[30017]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T11:42:00.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.129", "placement": "1;smithi129=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:00.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: Saving service iscsi.datapool spec with placement smithi129=iscsi.a;count:1 2024-09-17T11:42:00.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: pgmap v181: 100 pgs: 21 creating+peering, 48 unknown, 31 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-17T11:42:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T11:42:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:00 smithi129 ceph-mon[33364]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 -- 172.21.15.129:0/377532467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 msgr2=0x7f7080103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 --2- 172.21.15.129:0/377532467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080103fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7074009930 tx=0x7f707402a420 comp rx=0 tx=0).stop 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 -- 172.21.15.129:0/377532467 shutdown_connections 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 --2- 172.21.15.129:0/377532467 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080104520 0x7f708010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 --2- 172.21.15.129:0/377532467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080103fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 --2- 172.21.15.129:0/377532467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080102a00 0x7f7080102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 -- 172.21.15.129:0/377532467 >> 172.21.15.129:0/377532467 conn(0x7f70800fe2f0 msgr2=0x7f7080100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 -- 172.21.15.129:0/377532467 shutdown_connections 2024-09-17T11:42:01.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.043+0000 7f708564b640 1 -- 172.21.15.129:0/377532467 wait complete. 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 Processor -- start 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 -- start start 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080102a00 0x7f7080198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080104520 0x7f70801997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f708010dd30 con 0x7f7080104520 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f708010dbb0 con 0x7f7080103b60 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f708564b640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f708010deb0 con 0x7f7080102a00 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:01.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47090/0 (socket says 172.21.15.129:47090) 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 -- 172.21.15.129:0/1568271464 learned_addr learned my addr 172.21.15.129:0/1568271464 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707effd640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080102a00 0x7f7080198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707f7fe640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080104520 0x7f70801997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 -- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080102a00 msgr2=0x7f7080198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080102a00 0x7f7080198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 -- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080104520 msgr2=0x7f70801997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080104520 0x7f70801997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.044+0000 7f707e7fc640 1 -- 172.21.15.129:0/1568271464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708019a050 con 0x7f7080103b60 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f707e7fc640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7074005940 tx=0x7f7074032c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707403f030 con 0x7f7080103b60 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f7074032e30 con 0x7f7080103b60 2024-09-17T11:42:01.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708019a2e0 con 0x7f7080103b60 2024-09-17T11:42:01.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074035070 con 0x7f7080103b60 2024-09-17T11:42:01.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.045+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801a0a20 con 0x7f7080103b60 2024-09-17T11:42:01.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.046+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7080102e00 con 0x7f7080103b60 2024-09-17T11:42:01.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.046+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f7074042030 con 0x7f7080103b60 2024-09-17T11:42:01.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.047+0000 7f7067fff640 1 --2- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 0x7f706807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:01.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.047+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5915+0+0 (secure 0 0 0) 0x7f70740be1e0 con 0x7f7080103b60 2024-09-17T11:42:01.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.047+0000 7f707effd640 1 --2- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 0x7f706807d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:01.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.047+0000 7f707effd640 1 --2- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 0x7f706807d1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f707000bbe0 tx=0x7f707000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:01.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.051+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f7074089af0 con 0x7f7080103b60 2024-09-17T11:42:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.241+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}) -- 0x7f7080063590 con 0x7f706807ace0 2024-09-17T11:42:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.248+0000 7f7067fff640 1 -- 172.21.15.129:0/1568271464 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7f7080063590 con 0x7f706807ace0 2024-09-17T11:42:01.249 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled prometheus update... 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 msgr2=0x7f706807d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 0x7f706807d1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f707000bbe0 tx=0x7f707000b390 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 msgr2=0x7f7080198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7074005940 tx=0x7f7074032c40 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 shutdown_connections 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> v2:172.21.15.49:6800/1112797402 conn(0x7f706807ace0 0x7f706807d1a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7080104520 0x7f70801997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7080103b60 0x7f7080198b70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 --2- 172.21.15.129:0/1568271464 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7080102a00 0x7f7080198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 >> 172.21.15.129:0/1568271464 conn(0x7f70800fe2f0 msgr2=0x7f70800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 shutdown_connections 2024-09-17T11:42:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:01.250+0000 7f708564b640 1 -- 172.21.15.129:0/1568271464 wait complete. 2024-09-17T11:42:02.026 DEBUG:teuthology.orchestra.run.smithi129:prometheus.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T11:42:02.028 INFO:tasks.cephadm:Adding node-exporter.a on smithi049 2024-09-17T11:42:02.028 INFO:tasks.cephadm:Adding node-exporter.b on smithi129 2024-09-17T11:42:02.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi049=a;smithi129=b' 2024-09-17T11:42:02.312 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:42:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T11:42:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: Saving service prometheus spec with placement smithi129=a;count:1 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: Saving service prometheus spec with placement smithi129=a;count:1 2024-09-17T11:42:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:01 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: Saving service prometheus spec with placement smithi129=a;count:1 2024-09-17T11:42:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:01 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:02.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 -- 172.21.15.129:0/1263039567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 msgr2=0x7fcd20103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1263039567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20103fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009930 tx=0x7fcd1402a420 comp rx=0 tx=0).stop 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 -- 172.21.15.129:0/1263039567 shutdown_connections 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1263039567 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20104520 0x7fcd2010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1263039567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20103fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1263039567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20102a00 0x7fcd20102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 -- 172.21.15.129:0/1263039567 >> 172.21.15.129:0/1263039567 conn(0x7fcd200fe2f0 msgr2=0x7fcd20100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 -- 172.21.15.129:0/1263039567 shutdown_connections 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.769+0000 7fcd249aa640 1 -- 172.21.15.129:0/1263039567 wait complete. 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 Processor -- start 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 -- start start 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20102a00 0x7fcd20198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:02.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 0x7fcd20199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.770+0000 7fcd249aa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2010dd30 con 0x7fcd20104520 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd249aa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2010dbb0 con 0x7fcd20103b60 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd249aa640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2010deb0 con 0x7fcd20102a00 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43092/0 (socket says 172.21.15.129:43092) 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 -- 172.21.15.129:0/1409719321 learned_addr learned my addr 172.21.15.129:0/1409719321 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1e59c640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20102a00 0x7fcd20198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 -- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20102a00 msgr2=0x7fcd20198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20102a00 0x7fcd20198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1ed9d640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 0x7fcd20199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 -- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 msgr2=0x7fcd20199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:02.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 0x7fcd20199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 -- 172.21.15.129:0/1409719321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd20199fc0 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1ed9d640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 0x7fcd20199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd1dd9b640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005940 tx=0x7fcd14032c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1403f030 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fcd14032e30 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019a250 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14035070 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.771+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201a07c0 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.772+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd20102e00 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.773+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7fcd14042030 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.773+0000 7fcd0f7fe640 1 --2- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 0x7fcd0807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.773+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5926+0+0 (secure 0 0 0) 0x7fcd140be2c0 con 0x7fcd20103b60 2024-09-17T11:42:02.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.774+0000 7fcd1e59c640 1 --2- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 0x7fcd0807d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:02.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.774+0000 7fcd1e59c640 1 --2- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 0x7fcd0807d1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000abe0 tx=0x7fcd1000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:02.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.778+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fcd14085760 con 0x7fcd20103b60 2024-09-17T11:42:02.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.970+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi049=a;smithi129=b", "target": ["mon-mgr", ""]}) -- 0x7fcd20063590 con 0x7fcd0807ace0 2024-09-17T11:42:02.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.977+0000 7fcd0f7fe640 1 -- 172.21.15.129:0/1409719321 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fcd20063590 con 0x7fcd0807ace0 2024-09-17T11:42:02.978 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled node-exporter update... 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 msgr2=0x7fcd0807d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 0x7fcd0807d1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000abe0 tx=0x7fcd1000a2d0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 msgr2=0x7fcd20198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005940 tx=0x7fcd14032c40 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 shutdown_connections 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> v2:172.21.15.49:6800/1112797402 conn(0x7fcd0807ace0 0x7fcd0807d1a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd20104520 0x7fcd20199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd20103b60 0x7fcd20198ae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 --2- 172.21.15.129:0/1409719321 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcd20102a00 0x7fcd20198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 >> 172.21.15.129:0/1409719321 conn(0x7fcd200fe2f0 msgr2=0x7fcd200ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 shutdown_connections 2024-09-17T11:42:02.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:02.979+0000 7fcd249aa640 1 -- 172.21.15.129:0/1409719321 wait complete. 2024-09-17T11:42:03.359 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: pgmap v184: 132 pgs: 7 creating+peering, 50 unknown, 75 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.360 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:03.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.361 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:02 smithi129 ceph-mon[33364]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T11:42:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: pgmap v184: 132 pgs: 7 creating+peering, 50 unknown, 75 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-17T11:42:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[30017]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: pgmap v184: 132 pgs: 7 creating+peering, 50 unknown, 75 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1887620039' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3284909247' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T11:42:03.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:02 smithi049 ceph-mon[40223]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T11:42:03.680 DEBUG:teuthology.orchestra.run.smithi049:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T11:42:03.682 DEBUG:teuthology.orchestra.run.smithi129:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T11:42:03.684 INFO:tasks.cephadm:Adding alertmanager.a on smithi049 2024-09-17T11:42:03.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi049=a' 2024-09-17T11:42:04.503 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:42:04.623 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi049=a;smithi129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:04.623 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: Saving service node-exporter spec with placement smithi049=a;smithi129=b;count:2 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:04.624 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:04 smithi129 ceph-mon[33364]: Deploying daemon iscsi.iscsi.a on smithi129 2024-09-17T11:42:04.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi049=a;smithi129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:04.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: Saving service node-exporter spec with placement smithi049=a;smithi129=b;count:2 2024-09-17T11:42:04.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[30017]: Deploying daemon iscsi.iscsi.a on smithi129 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi049=a;smithi129=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: Saving service node-exporter spec with placement smithi049=a;smithi129=b;count:2 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:04.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-17T11:42:04.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T11:42:04.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:04.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:04 smithi049 ceph-mon[40223]: Deploying daemon iscsi.iscsi.a on smithi129 2024-09-17T11:42:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:05 smithi129 ceph-mon[33364]: pgmap v187: 132 pgs: 8 creating+peering, 21 unknown, 103 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:42:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:05 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:42:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:05 smithi129 ceph-mon[33364]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:42:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:05 smithi129 ceph-mon[33364]: Cluster is now healthy 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 -- 172.21.15.129:0/1443411195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 msgr2=0x7f6fe410aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 --2- 172.21.15.129:0/1443411195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe410aea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009cd0 tx=0x7f6fdc02a8e0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 -- 172.21.15.129:0/1443411195 shutdown_connections 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 --2- 172.21.15.129:0/1443411195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe410aea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 --2- 172.21.15.129:0/1443411195 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4103b60 0x7f6fe4103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 --2- 172.21.15.129:0/1443411195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4102a00 0x7f6fe4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 -- 172.21.15.129:0/1443411195 >> 172.21.15.129:0/1443411195 conn(0x7f6fe40fe2f0 msgr2=0x7f6fe4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 -- 172.21.15.129:0/1443411195 shutdown_connections 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.465+0000 7f6fec276640 1 -- 172.21.15.129:0/1443411195 wait complete. 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 Processor -- start 2024-09-17T11:42:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 -- start start 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4102a00 0x7f6fe4076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4103b60 0x7f6fe4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410dd30 con 0x7f6fe4103b60 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410dbb0 con 0x7f6fe4104520 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fec276640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe410deb0 con 0x7f6fe4102a00 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43116/0 (socket says 172.21.15.129:43116) 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 -- 172.21.15.129:0/2320521913 learned_addr learned my addr 172.21.15.129:0/2320521913 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea012640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4102a00 0x7f6fe4076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fe9811640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4103b60 0x7f6fe4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 -- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4102a00 msgr2=0x7f6fe4076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4102a00 0x7f6fe4076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 -- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4103b60 msgr2=0x7f6fe4075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4103b60 0x7f6fe4075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.466+0000 7f6fea813640 1 -- 172.21.15.129:0/2320521913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe4076340 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fea813640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0097b0 tx=0x7f6fdc003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004450 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe41a2480 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f6fdc0045f0 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc046480 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.467+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a29e0 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.468+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe4102e00 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.468+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f6fdc0305a0 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.469+0000 7f6fd77fe640 1 --2- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 0x7f6fd007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.469+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5942+0+0 (secure 0 0 0) 0x7f6fdc02e030 con 0x7f6fe4104520 2024-09-17T11:42:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.469+0000 7f6fea012640 1 --2- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 0x7f6fd007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.470+0000 7f6fea012640 1 --2- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 0x7f6fd007d1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000abe0 tx=0x7f6fe000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.473+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6fdc04b060 con 0x7f6fe4104520 2024-09-17T11:42:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[40223]: pgmap v187: 132 pgs: 8 creating+peering, 21 unknown, 103 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:42:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:42:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[40223]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:42:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[40223]: Cluster is now healthy 2024-09-17T11:42:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[30017]: pgmap v187: 132 pgs: 8 creating+peering, 21 unknown, 103 active+clean; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:42:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:42:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[30017]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T11:42:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:05 smithi049 ceph-mon[30017]: Cluster is now healthy 2024-09-17T11:42:05.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.694+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi049=a", "target": ["mon-mgr", ""]}) -- 0x7f6fe402e840 con 0x7f6fd007ace0 2024-09-17T11:42:05.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.703+0000 7f6fd77fe640 1 -- 172.21.15.129:0/2320521913 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7f6fe402e840 con 0x7f6fd007ace0 2024-09-17T11:42:05.705 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled alertmanager update... 2024-09-17T11:42:05.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 msgr2=0x7f6fd007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:05.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 0x7f6fd007d1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000abe0 tx=0x7f6fe000a2d0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 msgr2=0x7f6fe4075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0097b0 tx=0x7f6fdc003f10 comp rx=0 tx=0).stop 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 shutdown_connections 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> v2:172.21.15.49:6800/1112797402 conn(0x7f6fd007ace0 0x7f6fd007d1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fe4104520 0x7f6fe4075a80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fe4103b60 0x7f6fe4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 --2- 172.21.15.129:0/2320521913 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6fe4102a00 0x7f6fe4076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 >> 172.21.15.129:0/2320521913 conn(0x7f6fe40fe2f0 msgr2=0x7f6fe40ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 shutdown_connections 2024-09-17T11:42:05.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:05.706+0000 7f6fec276640 1 -- 172.21.15.129:0/2320521913 wait complete. 2024-09-17T11:42:06.387 DEBUG:teuthology.orchestra.run.smithi049:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T11:42:06.390 INFO:tasks.cephadm:Adding grafana.a on smithi129 2024-09-17T11:42:06.391 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi129=a' 2024-09-17T11:42:06.627 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 podman[60185]: 2024-09-17 11:42:06.503497571 +0000 UTC m=+0.019306703 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T11:42:06.627 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 podman[60185]: 2024-09-17 11:42:06.626521931 +0000 UTC m=+0.142331052 container create 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=main-7498493, CEPH_POINT_RELEASE=) 2024-09-17T11:42:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:06 smithi129 ceph-mon[33364]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi049=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:06 smithi129 ceph-mon[33364]: Saving service alertmanager spec with placement smithi049=a;count:1 2024-09-17T11:42:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:06 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:06 smithi129 ceph-mon[33364]: pgmap v188: 132 pgs: 8 creating+peering, 124 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.0 KiB/s wr, 130 op/s 2024-09-17T11:42:06.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 podman[60185]: 2024-09-17 11:42:06.870751623 +0000 UTC m=+0.386560756 container init 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T11:42:06.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 podman[60185]: 2024-09-17 11:42:06.875988505 +0000 UTC m=+0.391797627 container start 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[30017]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi049=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[30017]: Saving service alertmanager spec with placement smithi049=a;count:1 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[30017]: pgmap v188: 132 pgs: 8 creating+peering, 124 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.0 KiB/s wr, 130 op/s 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[40223]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi049=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:07.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[40223]: Saving service alertmanager spec with placement smithi049=a;count:1 2024-09-17T11:42:07.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:07.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:06 smithi049 ceph-mon[40223]: pgmap v188: 132 pgs: 8 creating+peering, 124 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.0 KiB/s wr, 130 op/s 2024-09-17T11:42:07.313 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 bash[60185]: 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 2024-09-17T11:42:07.313 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:06 smithi129 systemd[1]: Started Ceph iscsi.iscsi.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:42:07.314 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Started the configuration object watcher 2024-09-17T11:42:07.314 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Checking for config object changes every 1s 2024-09-17T11:42:07.314 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Processing osd blocklist entries for this node 2024-09-17T11:42:07.580 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:42:07.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Reading the configuration object to update local LIO configuration 2024-09-17T11:42:07.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Configuration does not have an entry for this host(smithi129.front.sepia.ceph.com) - nothing to define to LIO 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: * Environment: production 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: Use a production WSGI server instead. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: * Debug mode: off 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug * Running on all addresses. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: * Running on all addresses. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug * Running on http://[fe80::ec4:7aff:fe88:7261]:5000/ (Press CTRL+C to quit) 2024-09-17T11:42:07.702 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: * Running on http://[fe80::ec4:7aff:fe88:7261]:5000/ (Press CTRL+C to quit) 2024-09-17T11:42:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 -- 172.21.15.129:0/2993381973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654103430 msgr2=0x7f5654109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 --2- 172.21.15.129:0/2993381973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654103430 0x7f5654109db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f564c009cc0 tx=0x7f564c02a8a0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 -- 172.21.15.129:0/2993381973 shutdown_connections 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 --2- 172.21.15.129:0/2993381973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654103430 0x7f5654109db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 --2- 172.21.15.129:0/2993381973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654102a70 0x7f5654102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 --2- 172.21.15.129:0/2993381973 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654101910 0x7f5654101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 -- 172.21.15.129:0/2993381973 >> 172.21.15.129:0/2993381973 conn(0x7f56540fd200 msgr2=0x7f56540ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 -- 172.21.15.129:0/2993381973 shutdown_connections 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.065+0000 7f565a324640 1 -- 172.21.15.129:0/2993381973 wait complete. 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 Processor -- start 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 -- start start 2024-09-17T11:42:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654101910 0x7f5654197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654103430 0x7f56541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f565410cc80 con 0x7f5654101910 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f565410cb00 con 0x7f5654102a70 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.066+0000 7f565a324640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f565410ce00 con 0x7f5654103430 2024-09-17T11:42:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43210/0 (socket says 172.21.15.129:43210) 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 -- 172.21.15.129:0/3972438257 learned_addr learned my addr 172.21.15.129:0/3972438257 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56588c1640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654103430 0x7f56541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f5653fff640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654101910 0x7f5654197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 -- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654103430 msgr2=0x7f56541986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654103430 0x7f56541986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 -- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654101910 msgr2=0x7f5654197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654101910 0x7f5654197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 -- 172.21.15.129:0/3972438257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5654198f70 con 0x7f5654102a70 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56537fe640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f564800e9f0 tx=0x7f564800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800cd80 con 0x7f5654102a70 2024-09-17T11:42:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f56480108b0 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5654199260 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648018870 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.067+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565419f940 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.068+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5654101d10 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.069+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 103044+0+0 (secure 0 0 0) 0x7f5648018a10 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.069+0000 7f56517fa640 1 --2- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 0x7f563c07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.070+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5942+0+0 (secure 0 0 0) 0x7f564809e0e0 con 0x7f5654102a70 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.070+0000 7f5653fff640 1 --2- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 0x7f563c07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.070+0000 7f5653fff640 1 --2- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 0x7f563c07d1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f564400bbe0 tx=0x7f564400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:08.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.074+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f5648069a10 con 0x7f5654102a70 2024-09-17T11:42:08.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.142+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7f5648061250 con 0x7f5654102a70 2024-09-17T11:42:08.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.288+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 --> v2:172.21.15.49:6800/1112797402 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}) -- 0x7f565402e7f0 con 0x7f563c07ace0 2024-09-17T11:42:08.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T11:42:08.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.291 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.291 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: Deploying daemon prometheus.a on smithi129 2024-09-17T11:42:08.291 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/2793076983' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-17T11:42:08.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.295+0000 7f56517fa640 1 -- 172.21.15.129:0/3972438257 <== mgr.14152 v2:172.21.15.49:6800/1112797402 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f565402e7f0 con 0x7f563c07ace0 2024-09-17T11:42:08.296 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled grafana update... 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 msgr2=0x7f563c07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 0x7f563c07d1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f564400bbe0 tx=0x7f564400b390 comp rx=0 tx=0).stop 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 msgr2=0x7f5654197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f564800e9f0 tx=0x7f564800eeb0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 shutdown_connections 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> v2:172.21.15.49:6800/1112797402 conn(0x7f563c07ace0 0x7f563c07d1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f5654103430 0x7f56541986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5654102a70 0x7f5654197a90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 --2- 172.21.15.129:0/3972438257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5654101910 0x7f5654197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:08.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 >> 172.21.15.129:0/3972438257 conn(0x7f56540fd200 msgr2=0x7f56540fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:08.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 shutdown_connections 2024-09-17T11:42:08.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:08.297+0000 7f565a324640 1 -- 172.21.15.129:0/3972438257 wait complete. 2024-09-17T11:42:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: Deploying daemon prometheus.a on smithi129 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/2793076983' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: Deploying daemon prometheus.a on smithi129 2024-09-17T11:42:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/2793076983' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-17T11:42:09.253 DEBUG:teuthology.orchestra.run.smithi129:grafana.a> sudo journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T11:42:09.256 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T11:42:09.256 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T11:42:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[40223]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-09-17T11:42:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[40223]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T11:42:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[40223]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[40223]: Saving service grafana spec with placement smithi129=a;count:1 2024-09-17T11:42:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[30017]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-09-17T11:42:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[30017]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T11:42:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[30017]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[30017]: Saving service grafana spec with placement smithi129=a;count:1 2024-09-17T11:42:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:09 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:09 smithi129 ceph-mon[33364]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-09-17T11:42:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:09 smithi129 ceph-mon[33364]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T11:42:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:09 smithi129 ceph-mon[33364]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi129=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T11:42:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:09 smithi129 ceph-mon[33364]: Saving service grafana spec with placement smithi129=a;count:1 2024-09-17T11:42:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:09 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:09.523 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 -- 172.21.15.49:0/2157563523 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588104520 msgr2=0x7fb58810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 --2- 172.21.15.49:0/2157563523 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588104520 0x7fb58810aea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb580009d00 tx=0x7fb58002a910 comp rx=0 tx=0).stop 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 -- 172.21.15.49:0/2157563523 shutdown_connections 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 --2- 172.21.15.49:0/2157563523 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588104520 0x7fb58810aea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 --2- 172.21.15.49:0/2157563523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588103b60 0x7fb588103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 --2- 172.21.15.49:0/2157563523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588102a00 0x7fb588102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 -- 172.21.15.49:0/2157563523 >> 172.21.15.49:0/2157563523 conn(0x7fb5880fe2f0 msgr2=0x7fb588100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 -- 172.21.15.49:0/2157563523 shutdown_connections 2024-09-17T11:42:09.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.961+0000 7fb58cd25640 1 -- 172.21.15.49:0/2157563523 wait complete. 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 Processor -- start 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 -- start start 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588102a00 0x7fb5881984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 0x7fb588198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810dd80 con 0x7fb588104520 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810dc00 con 0x7fb588102a00 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58cd25640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fb58810df00 con 0x7fb588103b60 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb585d9b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 0x7fb588198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56700/0 (socket says 172.21.15.49:56700) 2024-09-17T11:42:09.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb586d9d640 1 -- 172.21.15.49:0/1472851701 learned_addr learned my addr 172.21.15.49:0/1472851701 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.962+0000 7fb58659c640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588102a00 0x7fb5881984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 -- 172.21.15.49:0/1472851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588102a00 msgr2=0x7fb5881984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588102a00 0x7fb5881984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 -- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 msgr2=0x7fb588199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 -- 172.21.15.49:0/1472851701 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58819a000 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb586d9d640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb585d9b640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 0x7fb588198b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005fd0 tx=0x7fb57c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c009c30 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fb57c0044b0 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58819a2f0 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c003d10 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.963+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a09d0 con 0x7fb588103b60 2024-09-17T11:42:09.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.964+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb588102e00 con 0x7fb588103b60 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.965+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7fb57c003eb0 con 0x7fb588103b60 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.965+0000 7fb5777fe640 1 --2- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 0x7fb57007d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.965+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fb57c09d8e0 con 0x7fb588103b60 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.965+0000 7fb58659c640 1 --2- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 0x7fb57007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.966+0000 7fb58659c640 1 --2- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 0x7fb57007d230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb578009bd0 tx=0x7fb578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:09.970+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fb57c017020 con 0x7fb588103b60 2024-09-17T11:42:10.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.230+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb5881095a0 con 0x7fb588103b60 2024-09-17T11:42:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.235+0000 7fb5777fe640 1 -- 172.21.15.49:0/1472851701 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7fb57c069050 con 0x7fb588103b60 2024-09-17T11:42:10.237 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-09-17T11:42:10.237 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQASa+lmrZrUDRAAT+roxhSa5xUh98RlHmeLVQ== 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.237+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 msgr2=0x7fb57007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.237+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 0x7fb57007d230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb578009bd0 tx=0x7fb578005f90 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.237+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 msgr2=0x7fb588198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.237+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 0x7fb588198b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005fd0 tx=0x7fb57c00dde0 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 shutdown_connections 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> v2:172.21.15.49:6800/1112797402 conn(0x7fb57007ad70 0x7fb57007d230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb588104520 0x7fb588199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fb588103b60 0x7fb588198b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 --2- 172.21.15.49:0/1472851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb588102a00 0x7fb5881984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 >> 172.21.15.49:0/1472851701 conn(0x7fb5880fe2f0 msgr2=0x7fb5880ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 shutdown_connections 2024-09-17T11:42:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:10.238+0000 7fb58cd25640 1 -- 172.21.15.49:0/1472851701 wait complete. 2024-09-17T11:42:10.555 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:10 smithi049 ceph-mon[40223]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T11:42:10.555 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:10 smithi049 ceph-mon[30017]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T11:42:10.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:10 smithi129 ceph-mon[33364]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T11:42:11.139 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:42:11.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T11:42:11.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T11:42:11.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T11:42:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[30017]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 143 op/s 2024-09-17T11:42:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1472851701' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[40223]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 143 op/s 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1472851701' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:11.438 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T11:42:11.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:11 smithi129 ceph-mon[33364]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 143 op/s 2024-09-17T11:42:11.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1472851701' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:11.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 -- 172.21.15.129:0/801274400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c1018e0 msgr2=0x7fa88c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 --2- 172.21.15.129:0/801274400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c1018e0 0x7fa88c101ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009930 tx=0x7fa87c02a420 comp rx=0 tx=0).stop 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 -- 172.21.15.129:0/801274400 shutdown_connections 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 --2- 172.21.15.129:0/801274400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c103400 0x7fa88c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 --2- 172.21.15.129:0/801274400 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c102a40 0x7fa88c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 --2- 172.21.15.129:0/801274400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c1018e0 0x7fa88c101ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.115+0000 7fa8925e8640 1 -- 172.21.15.129:0/801274400 >> 172.21.15.129:0/801274400 conn(0x7fa88c0fd1b0 msgr2=0x7fa88c0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.116+0000 7fa8925e8640 1 -- 172.21.15.129:0/801274400 shutdown_connections 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.116+0000 7fa8925e8640 1 -- 172.21.15.129:0/801274400 wait complete. 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.116+0000 7fa8925e8640 1 Processor -- start 2024-09-17T11:42:12.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.116+0000 7fa8925e8640 1 -- start start 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1018e0 0x7fa88c06b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c103400 0x7fa88c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10cc10 con 0x7fa88c1018e0 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10ca90 con 0x7fa88c102a40 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa8925e8640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c10cd90 con 0x7fa88c103400 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53202/0 (socket says 172.21.15.129:53202) 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 -- 172.21.15.129:0/3580759959 learned_addr learned my addr 172.21.15.129:0/3580759959 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88bfff640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1018e0 0x7fa88c06b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa890b85640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c103400 0x7fa88c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 -- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c103400 msgr2=0x7fa88c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:12.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c103400 0x7fa88c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 -- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1018e0 msgr2=0x7fa88c06b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1018e0 0x7fa88c06b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 -- 172.21.15.129:0/3580759959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c1a00a0 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.117+0000 7fa88b7fe640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d770 tx=0x7fa88000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.118+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880017070 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.118+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c1a0390 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.118+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c1a08d0 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.118+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fa8800054a0 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.118+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880004c40 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.119+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88c101ce0 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.120+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7fa880004de0 con 0x7fa88c102a40 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.121+0000 7fa8897fa640 1 --2- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 0x7fa87407d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:12.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.121+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fa88009d8a0 con 0x7fa88c102a40 2024-09-17T11:42:12.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.121+0000 7fa88bfff640 1 --2- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 0x7fa87407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:12.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.122+0000 7fa88bfff640 1 --2- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 0x7fa87407d230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009900 tx=0x7fa87c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:12.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.127+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fa880069090 con 0x7fa88c102a40 2024-09-17T11:42:12.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.479+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa88c02e810 con 0x7fa88c102a40 2024-09-17T11:42:12.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.484+0000 7fa8897fa640 1 -- 172.21.15.129:0/3580759959 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7fa88006df40 con 0x7fa88c102a40 2024-09-17T11:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-09-17T11:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAUa+lmgBuyHBAAq7Zv9tl/e69A89tAjYsTQQ== 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 msgr2=0x7fa87407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 0x7fa87407d230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009900 tx=0x7fa87c000bc0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 msgr2=0x7fa88c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d770 tx=0x7fa88000dc30 comp rx=0 tx=0).stop 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 shutdown_connections 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa87407ad70 0x7fa87407d230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa88c103400 0x7fa88c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa88c102a40 0x7fa88c19d1f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 --2- 172.21.15.129:0/3580759959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1018e0 0x7fa88c06b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:12.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 >> 172.21.15.129:0/3580759959 conn(0x7fa88c0fd1b0 msgr2=0x7fa88c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:12.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 shutdown_connections 2024-09-17T11:42:12.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T11:42:12.486+0000 7fa8925e8640 1 -- 172.21.15.129:0/3580759959 wait complete. 2024-09-17T11:42:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[40223]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 127 op/s 2024-09-17T11:42:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.129:0/3580759959' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[30017]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 127 op/s 2024-09-17T11:42:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.129:0/3580759959' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:13 smithi129 ceph-mon[33364]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 127 op/s 2024-09-17T11:42:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.129:0/3580759959' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-17T11:42:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T11:42:14.273 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T11:42:14.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T11:42:14.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T11:42:14.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-17T11:42:14.581 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:14.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 -- 172.21.15.49:0/42404426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8104540 msgr2=0x7f49a810cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/42404426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8104540 0x7f49a810cb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f499c009900 tx=0x7f499c02a410 comp rx=0 tx=0).stop 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 -- 172.21.15.49:0/42404426 shutdown_connections 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/42404426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a810d0f0 0x7f49a810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/42404426 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8104540 0x7f49a810cb20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/42404426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a8103b40 0x7f49a8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 -- 172.21.15.49:0/42404426 >> 172.21.15.49:0/42404426 conn(0x7f49a80fd710 msgr2=0x7f49a80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 -- 172.21.15.49:0/42404426 shutdown_connections 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.983+0000 7f49ad3bc640 1 -- 172.21.15.49:0/42404426 wait complete. 2024-09-17T11:42:14.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.984+0000 7f49ad3bc640 1 Processor -- start 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.984+0000 7f49ad3bc640 1 -- start start 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a8104540 0x7f49a8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 0x7f49a819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112480 con 0x7f49a810d0f0 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112300 con 0x7f49a8104540 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f49a8112600 con 0x7f49a8103b40 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:46746/0 (socket says 172.21.15.49:46746) 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 -- 172.21.15.49:0/364594506 learned_addr learned my addr 172.21.15.49:0/364594506 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a77fe640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 0x7f49a819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:14.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a67fc640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a8104540 0x7f49a8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 -- 172.21.15.49:0/364594506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a8104540 msgr2=0x7f49a8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a8104540 0x7f49a8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 -- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 msgr2=0x7f49a819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 0x7f49a819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 -- 172.21.15.49:0/364594506 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819eaf0 con 0x7f49a8103b40 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a77fe640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 0x7f49a819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49a6ffd640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f499800ea60 tx=0x7f499800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:14.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800cde0 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f49980035f0 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819ede0 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980044a0 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.985+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a54a0 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.986+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a8103f20 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.987+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7f4998004000 con 0x7f49a8103b40 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.988+0000 7f498ffff640 1 --2- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 0x7f499007d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:14.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.988+0000 7f49a67fc640 1 --2- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 0x7f499007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:14.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.988+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f499809e1a0 con 0x7f49a8103b40 2024-09-17T11:42:14.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.988+0000 7f49a67fc640 1 --2- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 0x7f499007d230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f499c031440 tx=0x7f499c005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:14.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:14.992+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f4998069910 con 0x7f49a8103b40 2024-09-17T11:42:15.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.177+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f49a8107850 con 0x7f49a8103b40 2024-09-17T11:42:15.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.178+0000 7f498ffff640 1 -- 172.21.15.49:0/364594506 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v25) ==== 86+0+187 (secure 0 0 0) 0x7f499806e7c0 con 0x7f49a8103b40 2024-09-17T11:42:15.179 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:15.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 msgr2=0x7f499007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:15.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 0x7f499007d230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f499c031440 tx=0x7f499c005b50 comp rx=0 tx=0).stop 2024-09-17T11:42:15.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 msgr2=0x7f49a8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f499800ea60 tx=0x7f499800ef20 comp rx=0 tx=0).stop 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 shutdown_connections 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> v2:172.21.15.49:6800/1112797402 conn(0x7f499007ad70 0x7f499007d230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49a810d0f0 0x7f49a819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49a8104540 0x7f49a8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 --2- 172.21.15.49:0/364594506 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f49a8103b40 0x7f49a8198a40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 >> 172.21.15.49:0/364594506 conn(0x7f49a80fd710 msgr2=0x7f49a8104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:15.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 shutdown_connections 2024-09-17T11:42:15.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:15.180+0000 7f49ad3bc640 1 -- 172.21.15.49:0/364594506 wait complete. 2024-09-17T11:42:15.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:15 smithi049 ceph-mon[40223]: pgmap v193: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-17T11:42:15.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/364594506' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-17T11:42:15.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:15 smithi049 ceph-mon[30017]: pgmap v193: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-17T11:42:15.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/364594506' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-17T11:42:15.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:15 smithi129 ceph-mon[33364]: pgmap v193: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-17T11:42:15.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/364594506' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-17T11:42:15.891 INFO:teuthology.orchestra.run.smithi049.stdout:[{"version":25,"timestamp":"2024-09-17T11:42:03.216416+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"foa5dPCssq6zvOkxj4WMu05pSgpXQFRb1EBalpN5"}]}] 2024-09-17T11:42:15.891 INFO:tasks.ceph_manager:config epoch is 25 2024-09-17T11:42:15.891 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T11:42:15.891 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T11:42:15.891 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T11:42:16.146 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 -- 172.21.15.49:0/1020606864 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1204075de0 msgr2=0x7f120410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 --2- 172.21.15.49:0/1020606864 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1204075de0 0x7f120410ee50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009900 tx=0x7f11f802a410 comp rx=0 tx=0).stop 2024-09-17T11:42:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 -- 172.21.15.49:0/1020606864 shutdown_connections 2024-09-17T11:42:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 --2- 172.21.15.49:0/1020606864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f1204111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 --2- 172.21.15.49:0/1020606864 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1204075de0 0x7f120410ee50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 --2- 172.21.15.49:0/1020606864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12040754c0 0x7f12040758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 -- 172.21.15.49:0/1020606864 >> 172.21.15.49:0/1020606864 conn(0x7f12040fe7d0 msgr2=0x7f1204100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 -- 172.21.15.49:0/1020606864 shutdown_connections 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.573+0000 7f120a552640 1 -- 172.21.15.49:0/1020606864 wait complete. 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 Processor -- start 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 -- start start 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12040754c0 0x7f120419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204075de0 0x7f120419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12041146d0 con 0x7f120410f390 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1204114550 con 0x7f1204075de0 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.574+0000 7f120a552640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f1204114850 con 0x7f12040754c0 2024-09-17T11:42:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59722/0 (socket says 172.21.15.49:59722) 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 -- 172.21.15.49:0/1709336468 learned_addr learned my addr 172.21.15.49:0/1709336468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f12037fe640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204075de0 0x7f120419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1203fff640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12040754c0 0x7f120419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 -- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12040754c0 msgr2=0x7f120419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12040754c0 0x7f120419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 -- 172.21.15.49:0/1709336468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204075de0 msgr2=0x7f120419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204075de0 0x7f120419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 -- 172.21.15.49:0/1709336468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12041a0cc0 con 0x7f120410f390 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f1208aef640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ec50 tx=0x7f11fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc010630 con 0x7f120410f390 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f11fc010c50 con 0x7f120410f390 2024-09-17T11:42:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12041a0fb0 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc017c10 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.575+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12041a7670 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.576+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12040758a0 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.577+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7f11fc0107d0 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.578+0000 7f12017fa640 1 --2- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 0x7f11ec07d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.578+0000 7f1203fff640 1 --2- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 0x7f11ec07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.578+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f11fc09e6b0 con 0x7f120410f390 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.578+0000 7f1203fff640 1 --2- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 0x7f11ec07d230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.582+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f11fc065a40 con 0x7f120410f390 2024-09-17T11:42:16.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.816+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1204109c20 con 0x7f120410f390 2024-09-17T11:42:16.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.819+0000 7f12017fa640 1 -- 172.21.15.49:0/1709336468 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) ==== 74+0+197725 (secure 0 0 0) 0x7f11fc069e20 con 0x7f120410f390 2024-09-17T11:42:16.823 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 msgr2=0x7f11ec07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:16.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 0x7f11ec07d230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f4005850 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 msgr2=0x7f12041a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ec50 tx=0x7f11fc00c6a0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 shutdown_connections 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> v2:172.21.15.49:6800/1112797402 conn(0x7f11ec07ad70 0x7f11ec07d230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120410f390 0x7f12041a04f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204075de0 0x7f120419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 --2- 172.21.15.49:0/1709336468 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f12040754c0 0x7f120419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 >> 172.21.15.49:0/1709336468 conn(0x7f12040fe7d0 msgr2=0x7f1204106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 shutdown_connections 2024-09-17T11:42:16.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:16.823+0000 7f120a552640 1 -- 172.21.15.49:0/1709336468 wait complete. 2024-09-17T11:42:17.429 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[40223]: pgmap v194: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-09-17T11:42:17.429 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1709336468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-17T11:42:17.430 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:17.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[30017]: pgmap v194: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-09-17T11:42:17.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1709336468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-17T11:42:17.430 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:17 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:17.431 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1112797402}]},"active_addr":"172.21.15.49:6800/1112797402","active_change":"2024-09-17T11:37:31.685898+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3963302993}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":789610672}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3728944021}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":628367473}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":684468509}]}]} 2024-09-17T11:42:17.435 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T11:42:17.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T11:42:17.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T11:42:17.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:42:17.481 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:17 smithi129 ceph-mon[33364]: pgmap v194: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-09-17T11:42:17.481 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1709336468' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-17T11:42:17.481 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:17 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:17.705 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 -- 172.21.15.49:0/3807018646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60107ff0 msgr2=0x7fbe6010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 --2- 172.21.15.49:0/3807018646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60107ff0 0x7fbe6010e7f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009ca0 tx=0x7fbe5802a8c0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 -- 172.21.15.49:0/3807018646 shutdown_connections 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 --2- 172.21.15.49:0/3807018646 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60107ff0 0x7fbe6010e7f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 --2- 172.21.15.49:0/3807018646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107670 0x7fbe60107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 --2- 172.21.15.49:0/3807018646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60101c60 0x7fbe60102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 -- 172.21.15.49:0/3807018646 >> 172.21.15.49:0/3807018646 conn(0x7fbe600fd6d0 msgr2=0x7fbe600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 -- 172.21.15.49:0/3807018646 shutdown_connections 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.140+0000 7fbe67f94640 1 -- 172.21.15.49:0/3807018646 wait complete. 2024-09-17T11:42:18.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 Processor -- start 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 -- start start 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 0x7fbe6019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107ff0 0x7fbe601a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60111380 con 0x7fbe60107670 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60111200 con 0x7fbe60107ff0 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe67f94640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fbe60111500 con 0x7fbe60101c60 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe6552f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe6552f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59734/0 (socket says 172.21.15.49:59734) 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe6552f640 1 -- 172.21.15.49:0/1268964967 learned_addr learned my addr 172.21.15.49:0/1268964967 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.141+0000 7fbe65d30640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 0x7fbe6019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe66531640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107ff0 0x7fbe601a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:18.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 -- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 msgr2=0x7fbe6019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 0x7fbe6019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 -- 172.21.15.49:0/1268964967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107ff0 msgr2=0x7fbe601a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107ff0 0x7fbe601a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 -- 172.21.15.49:0/1268964967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe601a0bc0 con 0x7fbe60107670 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe65d30640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 0x7fbe6019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe6552f640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d770 tx=0x7fbe5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50017070 con 0x7fbe60107670 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fbe500054a0 con 0x7fbe60107670 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe601a0eb0 con 0x7fbe60107670 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50004c40 con 0x7fbe60107670 2024-09-17T11:42:18.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.142+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601a7570 con 0x7fbe60107670 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.143+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe60102040 con 0x7fbe60107670 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.144+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7fbe50004de0 con 0x7fbe60107670 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.145+0000 7fbe56ffd640 1 --2- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 0x7fbe4c07d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.145+0000 7fbe65d30640 1 --2- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 0x7fbe4c07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.145+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fbe5009d8a0 con 0x7fbe60107670 2024-09-17T11:42:18.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.145+0000 7fbe65d30640 1 --2- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 0x7fbe4c07d230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0095f0 tx=0x7fbe5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:18.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.153+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbe50069090 con 0x7fbe60107670 2024-09-17T11:42:18.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.341+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe601a77d0 con 0x7fbe60107670 2024-09-17T11:42:18.342 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:18.343 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:18.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.343+0000 7fbe56ffd640 1 -- 172.21.15.49:0/1268964967 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+20716 (secure 0 0 0) 0x7fbe5006df40 con 0x7fbe60107670 2024-09-17T11:42:18.345 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:18.345 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":62,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","created":"2024-09-17T11:36:43.058933+0000","modified":"2024-09-17T11:42:09.218058+0000","last_up_change":"2024-09-17T11:41:46.960860+0000","last_in_change":"2024-09-17T11:41:25.982951+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T11:39:42.144376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-17T11:41:53.524013+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-17T11:41:54.398341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T11:41:56.054825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T11:41:57.998745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T11:42:00.054351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a304fc7-f010-473e-ad18-e04e9b886490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":501081646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":501081646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":501081646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6803","nonce":501081646}]},"public_addr":"172.21.15.49:6801/501081646","cluster_addr":"172.21.15.49:6802/501081646","heartbeat_back_addr":"172.21.15.49:6804/501081646","heartbeat_front_addr":"172.21.15.49:6803/501081646","state":["exists","up"]},{"osd":1,"uuid":"b1d4bec5-5437-46c5-b724-270c2867bd33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":1599533781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1599533781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1599533781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6807","nonce":1599533781}]},"public_addr":"172.21.15.49:6805/1599533781","cluster_addr":"172.21.15.49:6806/1599533781","heartbeat_back_addr":"172.21.15.49:6808/1599533781","heartbeat_front_addr":"172.21.15.49:6807/1599533781","state":["exists","up"]},{"osd":2,"uuid":"ef100460-beb3-49ed-b03c-764b2bd5c333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":185342269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":185342269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":185342269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6811","nonce":185342269}]},"public_addr":"172.21.15.49:6809/185342269","cluster_addr":"172.21.15.49:6810/185342269","heartbeat_back_addr":"172.21.15.49:6812/185342269","heartbeat_front_addr":"172.21.15.49:6811/185342269","state":["exists","up"]},{"osd":3,"uuid":"b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":1277715496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1277715496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1277715496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":1277715496}]},"public_addr":"172.21.15.49:6813/1277715496","cluster_addr":"172.21.15.49:6814/1277715496","heartbeat_back_addr":"172.21.15.49:6816/1277715496","heartbeat_front_addr":"172.21.15.49:6815/1277715496","state":["exists","up"]},{"osd":4,"uuid":"9500ec09-085d-45ae-8356-ea20c1a7d064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2156637849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":2156637849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2156637849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2156637849}]},"public_addr":"172.21.15.129:6800/2156637849","cluster_addr":"172.21.15.129:6801/2156637849","heartbeat_back_addr":"172.21.15.129:6803/2156637849","heartbeat_front_addr":"172.21.15.129:6802/2156637849","state":["exists","up"]},{"osd":5,"uuid":"a9691398-5234-4fb1-ba46-1369ac2b43cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2449624152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":2449624152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2449624152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2449624152}]},"public_addr":"172.21.15.129:6804/2449624152","cluster_addr":"172.21.15.129:6805/2449624152","heartbeat_back_addr":"172.21.15.129:6807/2449624152","heartbeat_front_addr":"172.21.15.129:6806/2449624152","state":["exists","up"]},{"osd":6,"uuid":"34249b34-b42f-4289-9797-11f3e971069e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3044459329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6809","nonce":3044459329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3044459329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3044459329}]},"public_addr":"172.21.15.129:6808/3044459329","cluster_addr":"172.21.15.129:6809/3044459329","heartbeat_back_addr":"172.21.15.129:6811/3044459329","heartbeat_front_addr":"172.21.15.129:6810/3044459329","state":["exists","up"]},{"osd":7,"uuid":"9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1293620562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6813","nonce":1293620562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1293620562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1293620562}]},"public_addr":"172.21.15.129:6812/1293620562","cluster_addr":"172.21.15.129:6813/1293620562","heartbeat_back_addr":"172.21.15.129:6815/1293620562","heartbeat_front_addr":"172.21.15.129:6814/1293620562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:38:47.278603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:13.199579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:39.193605+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:04.375752+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:27.617287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:54.268575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:20.430993+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:45.446088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6800/1315254490":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/2860709752":"2024-09-18T11:37:06.723518+0000","172.21.15.49:6800/1073451895":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3769210869":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/708267795":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3594659540":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/546102581":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/3939921930":"2024-09-18T11:37:06.723518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.345+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 msgr2=0x7fbe4c07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.345+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 0x7fbe4c07d230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0095f0 tx=0x7fbe5c005f90 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.345+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 msgr2=0x7fbe6019b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.345+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d770 tx=0x7fbe5000dc30 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 shutdown_connections 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbe4c07ad70 0x7fbe4c07d230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe60107ff0 0x7fbe601a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe60107670 0x7fbe6019b050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 --2- 172.21.15.49:0/1268964967 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbe60101c60 0x7fbe6019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:18.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 >> 172.21.15.49:0/1268964967 conn(0x7fbe600fd6d0 msgr2=0x7fbe6006ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:18.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 shutdown_connections 2024-09-17T11:42:18.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:18.346+0000 7fbe67f94640 1 -- 172.21.15.49:0/1268964967 wait complete. 2024-09-17T11:42:18.620 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:19.005 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T11:42:19.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T11:42:19.260 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-mon[33364]: pgmap v195: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:42:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1268964967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:19.427 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 systemd[1]: Starting Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:42:19.497 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:19 smithi049 ceph-mon[40223]: pgmap v195: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:42:19.497 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1268964967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:19.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:19 smithi049 ceph-mon[30017]: pgmap v195: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:42:19.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1268964967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:19.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 -- 172.21.15.49:0/2607490172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d10102da0 msgr2=0x7f3d10103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 --2- 172.21.15.49:0/2607490172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d10102da0 0x7f3d10103180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d0002a3f0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 -- 172.21.15.49:0/2607490172 shutdown_connections 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 --2- 172.21.15.49:0/2607490172 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10109130 0x7f3d1010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 --2- 172.21.15.49:0/2607490172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d101087b0 0x7f3d10108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 --2- 172.21.15.49:0/2607490172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d10102da0 0x7f3d10103180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 -- 172.21.15.49:0/2607490172 >> 172.21.15.49:0/2607490172 conn(0x7f3d100fe810 msgr2=0x7f3d10100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 -- 172.21.15.49:0/2607490172 shutdown_connections 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.680+0000 7f3d1524a640 1 -- 172.21.15.49:0/2607490172 wait complete. 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 Processor -- start 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 -- start start 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:19.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d10109130 0x7f3d1019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10112500 con 0x7f3d101087b0 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10112380 con 0x7f3d10109130 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.681+0000 7f3d1524a640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3d10112680 con 0x7f3d10102da0 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:46774/0 (socket says 172.21.15.49:46774) 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59752/0 (socket says 172.21.15.49:59752) 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0ed9d640 1 -- 172.21.15.49:0/3554906525 learned_addr learned my addr 172.21.15.49:0/3554906525 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0f59e640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d10109130 0x7f3d1019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 -- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 msgr2=0x7f3d101989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 -- 172.21.15.49:0/3554906525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d10109130 msgr2=0x7f3d1019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d10109130 0x7f3d1019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 -- 172.21.15.49:0/3554906525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1019ea80 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0ed9d640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d0e59c640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9f0 tx=0x7f3d0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400ccc0 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1019ed70 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3d040108b0 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04018660 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.682+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a5430 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.683+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d10103180 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.684+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 103095+0+0 (secure 0 0 0) 0x7f3d04010430 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.684+0000 7f3cf7fff640 1 --2- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 0x7f3cf807d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.685+0000 7f3d0ed9d640 1 --2- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 0x7f3cf807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.685+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f3d0409e030 con 0x7f3d101087b0 2024-09-17T11:42:19.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.685+0000 7f3d0ed9d640 1 --2- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 0x7f3cf807d230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d00031440 tx=0x7f3d000058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:19.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.689+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3d040697a0 con 0x7f3d101087b0 2024-09-17T11:42:19.775 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 podman[61222]: 2024-09-17 11:42:19.377972578 +0000 UTC m=+0.016792496 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-17T11:42:19.775 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 podman[61222]: 2024-09-17 11:42:19.529507273 +0000 UTC m=+0.168327189 container create 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:19.776 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 podman[61222]: 2024-09-17 11:42:19.774969993 +0000 UTC m=+0.413789910 container init 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:19.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.879+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d1010af10 con 0x7f3d101087b0 2024-09-17T11:42:19.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.880+0000 7f3cf7fff640 1 -- 172.21.15.49:0/3554906525 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+20716 (secure 0 0 0) 0x7f3d0406e650 con 0x7f3d101087b0 2024-09-17T11:42:19.881 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:19.882 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":62,"fsid":"f97d9e96-74e8-11ef-bceb-c7b262605968","created":"2024-09-17T11:36:43.058933+0000","modified":"2024-09-17T11:42:09.218058+0000","last_up_change":"2024-09-17T11:41:46.960860+0000","last_in_change":"2024-09-17T11:41:25.982951+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T11:39:42.144376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-17T11:41:53.524013+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-17T11:41:54.398341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-17T11:41:56.054825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T11:41:57.998745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T11:42:00.054351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a304fc7-f010-473e-ad18-e04e9b886490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":501081646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":501081646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":501081646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6803","nonce":501081646}]},"public_addr":"172.21.15.49:6801/501081646","cluster_addr":"172.21.15.49:6802/501081646","heartbeat_back_addr":"172.21.15.49:6804/501081646","heartbeat_front_addr":"172.21.15.49:6803/501081646","state":["exists","up"]},{"osd":1,"uuid":"b1d4bec5-5437-46c5-b724-270c2867bd33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":1599533781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1599533781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1599533781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6807","nonce":1599533781}]},"public_addr":"172.21.15.49:6805/1599533781","cluster_addr":"172.21.15.49:6806/1599533781","heartbeat_back_addr":"172.21.15.49:6808/1599533781","heartbeat_front_addr":"172.21.15.49:6807/1599533781","state":["exists","up"]},{"osd":2,"uuid":"ef100460-beb3-49ed-b03c-764b2bd5c333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":185342269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":185342269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":185342269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6811","nonce":185342269}]},"public_addr":"172.21.15.49:6809/185342269","cluster_addr":"172.21.15.49:6810/185342269","heartbeat_back_addr":"172.21.15.49:6812/185342269","heartbeat_front_addr":"172.21.15.49:6811/185342269","state":["exists","up"]},{"osd":3,"uuid":"b87f4dca-0b95-4b9c-adfa-f2e0f1ce0bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":1277715496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":1277715496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":1277715496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":1277715496}]},"public_addr":"172.21.15.49:6813/1277715496","cluster_addr":"172.21.15.49:6814/1277715496","heartbeat_back_addr":"172.21.15.49:6816/1277715496","heartbeat_front_addr":"172.21.15.49:6815/1277715496","state":["exists","up"]},{"osd":4,"uuid":"9500ec09-085d-45ae-8356-ea20c1a7d064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2156637849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":2156637849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2156637849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2156637849}]},"public_addr":"172.21.15.129:6800/2156637849","cluster_addr":"172.21.15.129:6801/2156637849","heartbeat_back_addr":"172.21.15.129:6803/2156637849","heartbeat_front_addr":"172.21.15.129:6802/2156637849","state":["exists","up"]},{"osd":5,"uuid":"a9691398-5234-4fb1-ba46-1369ac2b43cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2449624152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":2449624152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2449624152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2449624152}]},"public_addr":"172.21.15.129:6804/2449624152","cluster_addr":"172.21.15.129:6805/2449624152","heartbeat_back_addr":"172.21.15.129:6807/2449624152","heartbeat_front_addr":"172.21.15.129:6806/2449624152","state":["exists","up"]},{"osd":6,"uuid":"34249b34-b42f-4289-9797-11f3e971069e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3044459329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6809","nonce":3044459329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":3044459329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3044459329}]},"public_addr":"172.21.15.129:6808/3044459329","cluster_addr":"172.21.15.129:6809/3044459329","heartbeat_back_addr":"172.21.15.129:6811/3044459329","heartbeat_front_addr":"172.21.15.129:6810/3044459329","state":["exists","up"]},{"osd":7,"uuid":"9eceb3fc-8bc5-43c2-9bfa-99b6f2f197c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1293620562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6813","nonce":1293620562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1293620562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1293620562}]},"public_addr":"172.21.15.129:6812/1293620562","cluster_addr":"172.21.15.129:6813/1293620562","heartbeat_back_addr":"172.21.15.129:6815/1293620562","heartbeat_front_addr":"172.21.15.129:6814/1293620562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:38:47.278603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:13.199579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:39:39.193605+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:04.375752+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:27.617287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:40:54.268575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:20.430993+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T11:41:45.446088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6800/1315254490":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/2860709752":"2024-09-18T11:37:06.723518+0000","172.21.15.49:6800/1073451895":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3769210869":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/708267795":"2024-09-18T11:37:06.723518+0000","172.21.15.49:0/3594659540":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/546102581":"2024-09-18T11:37:31.685753+0000","172.21.15.49:0/3939921930":"2024-09-18T11:37:06.723518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.882+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 msgr2=0x7f3cf807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.882+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 0x7f3cf807d230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d00031440 tx=0x7f3d000058d0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 msgr2=0x7f3d10198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9f0 tx=0x7f3d0400edf0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 shutdown_connections 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> v2:172.21.15.49:6800/1112797402 conn(0x7f3cf807ad70 0x7f3cf807d230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d10109130 0x7f3d1019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d101087b0 0x7f3d10198f10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 --2- 172.21.15.49:0/3554906525 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3d10102da0 0x7f3d101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:19.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 >> 172.21.15.49:0/3554906525 conn(0x7f3d100fe810 msgr2=0x7f3d10100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:19.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 shutdown_connections 2024-09-17T11:42:19.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:19.883+0000 7f3d1524a640 1 -- 172.21.15.49:0/3554906525 wait complete. 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 podman[61222]: 2024-09-17 11:42:19.780007529 +0000 UTC m=+0.418827438 container start 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.799Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.800Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.800Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi129 (none))" 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.800Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.800Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.801Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.801Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-17T11:42:20.177 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.803Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.803Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.805Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.805Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.752µs 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.805Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.805Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.805Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.168µs wal_replay_duration=176.486µs wbl_replay_duration=136ns total_replay_duration=216.753µs 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.806Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.806Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.806Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.822Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.807669ms db_storage=851ns remote_storage=1.08µs web_handler=385ns query_engine=562ns scrape=479.568µs scrape_sd=89.321µs notify=710ns notify_sd=575ns rules=14.927009ms tracing=5.778µs 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.822Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:42:19.822Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 bash[61222]: 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a 2024-09-17T11:42:20.178 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:42:19 smithi129 systemd[1]: Started Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:42:20.459 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3554906525' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:20.459 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3554906525' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:20 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-17T11:42:20.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T11:42:20.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T11:42:20.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T11:42:20.490 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-17T11:42:20.490 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-17T11:42:20.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-17T11:42:20.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-17T11:42:20.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-17T11:42:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3554906525' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-17T11:42:20.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' 2024-09-17T11:42:20.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:20 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-17T11:42:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.050 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.052 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.057 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.057 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:21.246 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: ignoring --setuser ceph since I am not root 2024-09-17T11:42:21.246 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: ignoring --setgroup ceph since I am not root 2024-09-17T11:42:21.246 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-mgr[35051]: -- 172.21.15.129:0/3667597465 <== mon.2 v2:172.21.15.49:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bde3890e0 con 0x562bde367400 2024-09-17T11:42:21.246 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setuser ceph since I am not root 2024-09-17T11:42:21.246 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ignoring --setgroup ceph since I am not root 2024-09-17T11:42:21.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[30017]: pgmap v196: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 97 B/s wr, 1 op/s 2024-09-17T11:42:21.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[30017]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T11:42:21.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[30017]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T11:42:21.506 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[40223]: pgmap v196: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 97 B/s wr, 1 op/s 2024-09-17T11:42:21.507 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[40223]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T11:42:21.507 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:21 smithi049 ceph-mon[40223]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T11:42:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-mon[33364]: pgmap v196: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 97 B/s wr, 1 op/s 2024-09-17T11:42:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-mon[33364]: from='mgr.14152 172.21.15.49:0/4288096228' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T11:42:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:21 smithi129 ceph-mon[33364]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/2927344116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c107ff0 msgr2=0x7fbf9c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/2927344116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c107ff0 0x7fbf9c10e7f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009ca0 tx=0x7fbf9402a8c0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/2927344116 shutdown_connections 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/2927344116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c107ff0 0x7fbf9c10e7f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/2927344116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107670 0x7fbf9c107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/2927344116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c101c60 0x7fbf9c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/2927344116 >> 172.21.15.49:0/2927344116 conn(0x7fbf9c0fd6d0 msgr2=0x7fbf9c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/2927344116 shutdown_connections 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.721+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/2927344116 wait complete. 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 Processor -- start 2024-09-17T11:42:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 -- start start 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 0x7fbf9c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c107670 0x7fbf9c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107ff0 0x7fbf9c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1113c0 con 0x7fbf9c107ff0 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c111240 con 0x7fbf9c107670 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbfa1a0a640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c111540 con 0x7fbf9c101c60 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.722+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 0x7fbf9c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107ff0 0x7fbf9c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 0x7fbf9c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:46794/0 (socket says 172.21.15.49:46794) 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 -- 172.21.15.49:0/3081003557 learned_addr learned my addr 172.21.15.49:0/3081003557 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9a7fc640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c107670 0x7fbf9c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 -- 172.21.15.49:0/3081003557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c107670 msgr2=0x7fbf9c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c107670 0x7fbf9c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 -- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107ff0 msgr2=0x7fbf9c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107ff0 0x7fbf9c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 -- 172.21.15.49:0/3081003557 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c1a4b10 con 0x7fbf9c101c60 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9b7fe640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf9c107ff0 0x7fbf9c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbf9affd640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 0x7fbf9c199730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e9f0 tx=0x7fbf8c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00cd80 con 0x7fbf9c101c60 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fbf8c0054a0 con 0x7fbf9c101c60 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.723+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c010430 con 0x7fbf9c101c60 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.724+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c1a4e00 con 0x7fbf9c101c60 2024-09-17T11:42:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.724+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a5200 con 0x7fbf9c101c60 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.725+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf9c063860 con 0x7fbf9c101c60 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.725+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7fbf8c005020 con 0x7fbf9c101c60 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbfa0a08640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf8407adc0 0x7fbf8407d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbf9a7fc640 1 -- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf8407adc0 msgr2=0x7fbf8407d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fbf8c09e0f0 con 0x7fbf9c101c60 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbf9a7fc640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf8407adc0 0x7fbf8407d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbfa0a08640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6809/185342269 conn(0x7fbf84084410 0x7fbf84086870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 --> v2:172.21.15.49:6809/185342269 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf84086f00 con 0x7fbf84084410 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c0a5050 con 0x7fbf9c101c60 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.726+0000 7fbf9b7fe640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6809/185342269 conn(0x7fbf84084410 0x7fbf84086870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.727+0000 7fbf9b7fe640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6809/185342269 conn(0x7fbf84084410 0x7fbf84086870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:21.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.727+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== osd.2 v2:172.21.15.49:6809/185342269 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf84086f00 con 0x7fbf84084410 2024-09-17T11:42:21.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.756+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 --> v2:172.21.15.49:6809/185342269 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf9c063c60 con 0x7fbf84084410 2024-09-17T11:42:21.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa0a08640 1 -- 172.21.15.49:0/3081003557 <== osd.2 v2:172.21.15.49:6809/185342269 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf9c063c60 con 0x7fbf84084410 2024-09-17T11:42:21.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6809/185342269 conn(0x7fbf84084410 msgr2=0x7fbf84086870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:21.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6809/185342269 conn(0x7fbf84084410 0x7fbf84086870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf8407adc0 msgr2=0x7fbf8407d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/3081003557 >> v2:172.21.15.49:6800/1112797402 conn(0x7fbf8407adc0 0x7fbf8407d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 msgr2=0x7fbf9c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 --2- 172.21.15.49:0/3081003557 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbf9c101c60 0x7fbf9c199730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e9f0 tx=0x7fbf8c00edf0 comp rx=0 tx=0).stop 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbf9b7fe640 1 -- 172.21.15.49:0/3081003557 reap_dead start 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 shutdown_connections 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 >> 172.21.15.49:0/3081003557 conn(0x7fbf9c0fd6d0 msgr2=0x7fbf9c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 shutdown_connections 2024-09-17T11:42:21.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:21.757+0000 7fbfa1a0a640 1 -- 172.21.15.49:0/3081003557 wait complete. 2024-09-17T11:42:22.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 -- 172.21.15.49:0/56624704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f028810d130 msgr2=0x7f028810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 --2- 172.21.15.49:0/56624704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f028810d130 0x7f028810f560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0280009ca0 tx=0x7f028002a8c0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 -- 172.21.15.49:0/56624704 shutdown_connections 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 --2- 172.21.15.49:0/56624704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f028810d130 0x7f028810f560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 --2- 172.21.15.49:0/56624704 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288104580 0x7f028810cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 --2- 172.21.15.49:0/56624704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0288103b80 0x7f0288103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 -- 172.21.15.49:0/56624704 >> 172.21.15.49:0/56624704 conn(0x7f02880fd730 msgr2=0x7f02880ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 -- 172.21.15.49:0/56624704 shutdown_connections 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.153+0000 7f028c976640 1 -- 172.21.15.49:0/56624704 wait complete. 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 Processor -- start 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 -- start start 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 0x7f0288198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 0x7f0288198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028810d130 0x7f028819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:22.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.154+0000 7f028c976640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0288112480 con 0x7f0288104580 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028c976640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0288112300 con 0x7f028810d130 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028c976640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f0288112600 con 0x7f0288103b80 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028659c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 0x7f0288198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 0x7f0288198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 0x7f0288198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59798/0 (socket says 172.21.15.49:59798) 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028659c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 0x7f0288198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:46824/0 (socket says 172.21.15.49:46824) 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 -- 172.21.15.49:0/282703076 learned_addr learned my addr 172.21.15.49:0/282703076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0286d9d640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028810d130 0x7f028819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 -- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 msgr2=0x7f0288198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 0x7f0288198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 -- 172.21.15.49:0/282703076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028810d130 msgr2=0x7f028819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028810d130 0x7f028819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 -- 172.21.15.49:0/282703076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028819e9c0 con 0x7f0288104580 2024-09-17T11:42:22.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028659c640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0288103b80 0x7f0288198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0286d9d640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028810d130 0x7f028819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f0285d9b640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 0x7f0288198e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cd80 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f027c0108b0 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c018870 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.155+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028819ecb0 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.156+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881a5370 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.156+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02880ffd20 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.157+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7f027c010430 con 0x7f0288104580 2024-09-17T11:42:22.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f02777fe640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6800/1112797402 conn(0x7f027007adc0 0x7f027007d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f028659c640 1 -- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6800/1112797402 conn(0x7f027007adc0 msgr2=0x7f027007d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f027c09ea30 con 0x7f0288104580 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f028659c640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6800/1112797402 conn(0x7f027007adc0 0x7f027007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f02777fe640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6813/1277715496 conn(0x7f0270084390 0x7f02700867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 --> v2:172.21.15.49:6813/1277715496 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f027c028ea0 con 0x7f0270084390 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f027c09ef10 con 0x7f0288104580 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.158+0000 7f0286d9d640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6813/1277715496 conn(0x7f0270084390 0x7f02700867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.159+0000 7f0286d9d640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6813/1277715496 conn(0x7f0270084390 0x7f02700867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.160+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== osd.3 v2:172.21.15.49:6813/1277715496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f027c028ea0 con 0x7f0270084390 2024-09-17T11:42:22.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 --> v2:172.21.15.49:6813/1277715496 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f028806b900 con 0x7f0270084390 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f02777fe640 1 -- 172.21.15.49:0/282703076 <== osd.3 v2:172.21.15.49:6813/1277715496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f028806b900 con 0x7f0270084390 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6813/1277715496 conn(0x7f0270084390 msgr2=0x7f02700867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6813/1277715496 conn(0x7f0270084390 0x7f02700867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6800/1112797402 conn(0x7f027007adc0 msgr2=0x7f027007d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 --2- 172.21.15.49:0/282703076 >> v2:172.21.15.49:6800/1112797402 conn(0x7f027007adc0 0x7f027007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 msgr2=0x7f0288198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f028c976640 1 --2- 172.21.15.49:0/282703076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0288104580 0x7f0288198e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00eeb0 comp rx=0 tx=0).stop 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.174+0000 7f0286d9d640 1 -- 172.21.15.49:0/282703076 reap_dead start 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.175+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 shutdown_connections 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.175+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 >> 172.21.15.49:0/282703076 conn(0x7f02880fd730 msgr2=0x7f02880fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.175+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 shutdown_connections 2024-09-17T11:42:22.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:22.175+0000 7f028c976640 1 -- 172.21.15.49:0/282703076 wait complete. 2024-09-17T11:42:22.894 INFO:teuthology.orchestra.run.smithi049.stdout:111669149725 2024-09-17T11:42:22.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-17T11:42:23.424 INFO:teuthology.orchestra.run.smithi049.stdout:77309411362 2024-09-17T11:42:23.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T11:42:24.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 -- 172.21.15.49:0/2274191899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8998103c70 msgr2=0x7f8998104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 --2- 172.21.15.49:0/2274191899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8998103c70 0x7f8998104050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8988009900 tx=0x7f898802a410 comp rx=0 tx=0).stop 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 -- 172.21.15.49:0/2274191899 shutdown_connections 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 --2- 172.21.15.49:0/2274191899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f899810d0d0 0x7f899810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 --2- 172.21.15.49:0/2274191899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f899810ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 --2- 172.21.15.49:0/2274191899 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f8998103c70 0x7f8998104050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 -- 172.21.15.49:0/2274191899 >> 172.21.15.49:0/2274191899 conn(0x7f89980fd6d0 msgr2=0x7f89980ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 -- 172.21.15.49:0/2274191899 shutdown_connections 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.336+0000 7f899d202640 1 -- 172.21.15.49:0/2274191899 wait complete. 2024-09-17T11:42:24.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.337+0000 7f899d202640 1 Processor -- start 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.337+0000 7f899d202640 1 -- start start 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8998103c70 0x7f89981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f8998198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f899810d0d0 0x7f899819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112450 con 0x7f8998104590 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89981122d0 con 0x7f8998103c70 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899d202640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f89981125d0 con 0x7f899810d0d0 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f8998198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899759e640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f899810d0d0 0x7f899819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f8998198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51964/0 (socket says 172.21.15.49:51964) 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 -- 172.21.15.49:0/1095142793 learned_addr learned my addr 172.21.15.49:0/1095142793 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f8996d9d640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8998103c70 0x7f89981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 -- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f899810d0d0 msgr2=0x7f899819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f899810d0d0 0x7f899819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 -- 172.21.15.49:0/1095142793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8998103c70 msgr2=0x7f89981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8998103c70 0x7f89981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899659c640 1 -- 172.21.15.49:0/1095142793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819ea50 con 0x7f8998104590 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f899759e640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f899810d0d0 0x7f899819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.338+0000 7f8996d9d640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8998103c70 0x7f89981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f899659c640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f8998198ee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9c0 tx=0x7f898c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00cd80 con 0x7f8998104590 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f898c0096c0 con 0x7f8998104590 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c018700 con 0x7f8998104590 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819ed40 con 0x7f8998104590 2024-09-17T11:42:24.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.339+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a5400 con 0x7f8998104590 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.340+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8998063860 con 0x7f8998104590 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.341+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7f898c01f020 con 0x7f8998104590 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.341+0000 7f897ffff640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6800/1112797402 conn(0x7f898007adc0 0x7f898007d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.341+0000 7f8996d9d640 1 -- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6800/1112797402 conn(0x7f898007adc0 msgr2=0x7f898007d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.341+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f898c09e6b0 con 0x7f8998104590 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.341+0000 7f8996d9d640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6800/1112797402 conn(0x7f898007adc0 0x7f898007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.342+0000 7f897ffff640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6805/1599533781 conn(0x7f8980084390 0x7f89800867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.342+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 --> v2:172.21.15.49:6805/1599533781 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f898c019ec0 con 0x7f8980084390 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.342+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f898c09eb90 con 0x7f8998104590 2024-09-17T11:42:24.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.342+0000 7f899759e640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6805/1599533781 conn(0x7f8980084390 0x7f89800867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.342+0000 7f899759e640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6805/1599533781 conn(0x7f8980084390 0x7f89800867f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:24.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.343+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== osd.1 v2:172.21.15.49:6805/1599533781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f898c019ec0 con 0x7f8980084390 2024-09-17T11:42:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 --> v2:172.21.15.49:6805/1599533781 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8998063c60 con 0x7f8980084390 2024-09-17T11:42:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f897ffff640 1 -- 172.21.15.49:0/1095142793 <== osd.1 v2:172.21.15.49:6805/1599533781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8998063c60 con 0x7f8980084390 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6805/1599533781 conn(0x7f8980084390 msgr2=0x7f89800867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6805/1599533781 conn(0x7f8980084390 0x7f89800867f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6800/1112797402 conn(0x7f898007adc0 msgr2=0x7f898007d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 --2- 172.21.15.49:0/1095142793 >> v2:172.21.15.49:6800/1112797402 conn(0x7f898007adc0 0x7f898007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 msgr2=0x7f8998198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 --2- 172.21.15.49:0/1095142793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8998104590 0x7f8998198ee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9c0 tx=0x7f898c00edc0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899759e640 1 -- 172.21.15.49:0/1095142793 reap_dead start 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 shutdown_connections 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.357+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 >> 172.21.15.49:0/1095142793 conn(0x7f89980fd6d0 msgr2=0x7f89980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.358+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 shutdown_connections 2024-09-17T11:42:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.358+0000 7f899d202640 1 -- 172.21.15.49:0/1095142793 wait complete. 2024-09-17T11:42:24.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 -- 172.21.15.49:0/3103490850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 msgr2=0x7fa0f0108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 --2- 172.21.15.49:0/3103490850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f0108bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009900 tx=0x7fa0e402a410 comp rx=0 tx=0).stop 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 -- 172.21.15.49:0/3103490850 shutdown_connections 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 --2- 172.21.15.49:0/3103490850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 0x7fa0f010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 --2- 172.21.15.49:0/3103490850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f0108bc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 --2- 172.21.15.49:0/3103490850 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 0x7fa0f0103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 -- 172.21.15.49:0/3103490850 >> 172.21.15.49:0/3103490850 conn(0x7fa0f00fe820 msgr2=0x7fa0f0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 -- 172.21.15.49:0/3103490850 shutdown_connections 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.668+0000 7fa0f8654640 1 -- 172.21.15.49:0/3103490850 wait complete. 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 Processor -- start 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 -- start start 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 0x7fa0f019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 0x7fa0f01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01124b0 con 0x7fa0f0108780 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f0112330 con 0x7fa0f0109100 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f8654640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f0112630 con 0x7fa0f0102d70 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f63f0640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 0x7fa0f019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51992/0 (socket says 172.21.15.49:51992) 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 -- 172.21.15.49:0/712812089 learned_addr learned my addr 172.21.15.49:0/712812089 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f6bf1640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 0x7fa0f01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 -- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 msgr2=0x7fa0f019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 0x7fa0f019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 -- 172.21.15.49:0/712812089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 msgr2=0x7fa0f01a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.669+0000 7fa0f5bef640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 0x7fa0f01a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f5bef640 1 -- 172.21.15.49:0/712812089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f01a0c00 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f63f0640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fa0f0102d70 0x7fa0f019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f6bf1640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0f0109100 0x7fa0f01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f5bef640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f019b090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005ab0 tx=0x7fa0e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e403f070 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fa0e4005590 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4035030 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f01a0e90 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.670+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f01a75b0 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.671+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0f0063820 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.672+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7fa0e4042070 con 0x7fa0f0108780 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.672+0000 7fa0e37fe640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa0dc07adc0 0x7fa0dc07d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.672+0000 7fa0f63f0640 1 -- 172.21.15.49:0/712812089 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa0dc07adc0 msgr2=0x7fa0dc07d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fa0e40be6c0 con 0x7fa0f0108780 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0f63f0640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa0dc07adc0 0x7fa0dc07d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0e37fe640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.129:6812/1293620562 conn(0x7fa0dc084410 0x7fa0dc086870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 --> v2:172.21.15.129:6812/1293620562 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0e4049e50 con 0x7fa0dc084410 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fa0e40c5050 con 0x7fa0f0108780 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0f6bf1640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.129:6812/1293620562 conn(0x7fa0dc084410 0x7fa0dc086870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.673+0000 7fa0f6bf1640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.129:6812/1293620562 conn(0x7fa0dc084410 0x7fa0dc086870 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:24.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.674+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== osd.7 v2:172.21.15.129:6812/1293620562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0e4049e50 con 0x7fa0dc084410 2024-09-17T11:42:24.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.688+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 --> v2:172.21.15.129:6812/1293620562 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0f0063c20 con 0x7fa0dc084410 2024-09-17T11:42:24.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0e37fe640 1 -- 172.21.15.49:0/712812089 <== osd.7 v2:172.21.15.129:6812/1293620562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0f0063c20 con 0x7fa0dc084410 2024-09-17T11:42:24.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 >> v2:172.21.15.129:6812/1293620562 conn(0x7fa0dc084410 msgr2=0x7fa0dc086870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.129:6812/1293620562 conn(0x7fa0dc084410 0x7fa0dc086870 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa0dc07adc0 msgr2=0x7fa0dc07d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 --2- 172.21.15.49:0/712812089 >> v2:172.21.15.49:6800/1112797402 conn(0x7fa0dc07adc0 0x7fa0dc07d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 msgr2=0x7fa0f019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 --2- 172.21.15.49:0/712812089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0f0108780 0x7fa0f019b090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005ab0 tx=0x7fa0e40045e0 comp rx=0 tx=0).stop 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f6bf1640 1 -- 172.21.15.49:0/712812089 reap_dead start 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 shutdown_connections 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 >> 172.21.15.49:0/712812089 conn(0x7fa0f00fe820 msgr2=0x7fa0f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 shutdown_connections 2024-09-17T11:42:24.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.689+0000 7fa0f8654640 1 -- 172.21.15.49:0/712812089 wait complete. 2024-09-17T11:42:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 -- 172.21.15.49:0/3980603322 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184101c60 msgr2=0x7fe184102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 --2- 172.21.15.49:0/3980603322 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184101c60 0x7fe184102040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe174009860 tx=0x7fe17402a340 comp rx=0 tx=0).stop 2024-09-17T11:42:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 -- 172.21.15.49:0/3980603322 shutdown_connections 2024-09-17T11:42:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 --2- 172.21.15.49:0/3980603322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe184107ff0 0x7fe18410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 --2- 172.21.15.49:0/3980603322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184107670 0x7fe184107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 --2- 172.21.15.49:0/3980603322 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184101c60 0x7fe184102040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 -- 172.21.15.49:0/3980603322 >> 172.21.15.49:0/3980603322 conn(0x7fe1840fd6d0 msgr2=0x7fe1840ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.477+0000 7fe1888fe640 1 -- 172.21.15.49:0/3980603322 shutdown_connections 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 -- 172.21.15.49:0/3980603322 wait complete. 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 Processor -- start 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 -- start start 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184101c60 0x7fe184198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe184107670 0x7fe184198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 0x7fe18419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe184111380 con 0x7fe184101c60 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.478+0000 7fe1888fe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe184111200 con 0x7fe184107670 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe1888fe640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fe184111500 con 0x7fe184107ff0 2024-09-17T11:42:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 0x7fe18419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe18259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184101c60 0x7fe184198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 0x7fe18419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:52564/0 (socket says 172.21.15.49:52564) 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 -- 172.21.15.49:0/2923138303 learned_addr learned my addr 172.21.15.49:0/2923138303 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe17bfff640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe184107670 0x7fe184198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 -- 172.21.15.49:0/2923138303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe184107670 msgr2=0x7fe184198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe184107670 0x7fe184198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 -- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184101c60 msgr2=0x7fe184198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184101c60 0x7fe184198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 -- 172.21.15.49:0/2923138303 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18419ead0 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe18259c640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe184101c60 0x7fe184198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe182d9d640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 0x7fe18419e300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009fd0 tx=0x7fe17c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017070 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fe17c003df0 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18419edc0 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.479+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c010460 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.480+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1841a5480 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.481+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe184063860 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.481+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7fe17c003970 con 0x7fe184107ff0 2024-09-17T11:42:25.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17b7fe640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe17007adc0 0x7fe17007d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe18259c640 1 -- 172.21.15.49:0/2923138303 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe17007adc0 msgr2=0x7fe17007d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7fe17c09ded0 con 0x7fe184107ff0 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe18259c640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe17007adc0 0x7fe17007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17b7fe640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.129:6804/2449624152 conn(0x7fe170084390 0x7fe1700867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 --> v2:172.21.15.129:6804/2449624152 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe170086e80 con 0x7fe170084390 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe17c0a0620 con 0x7fe184107ff0 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.482+0000 7fe17bfff640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.129:6804/2449624152 conn(0x7fe170084390 0x7fe1700867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.483+0000 7fe17bfff640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.129:6804/2449624152 conn(0x7fe170084390 0x7fe1700867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.483+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== osd.5 v2:172.21.15.129:6804/2449624152 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe170086e80 con 0x7fe170084390 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.497+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 --> v2:172.21.15.129:6804/2449624152 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe184063c60 con 0x7fe170084390 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.497+0000 7fe17b7fe640 1 -- 172.21.15.49:0/2923138303 <== osd.5 v2:172.21.15.129:6804/2449624152 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe184063c60 con 0x7fe170084390 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 >> v2:172.21.15.129:6804/2449624152 conn(0x7fe170084390 msgr2=0x7fe1700867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.129:6804/2449624152 conn(0x7fe170084390 0x7fe1700867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe17007adc0 msgr2=0x7fe17007d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 --2- 172.21.15.49:0/2923138303 >> v2:172.21.15.49:6800/1112797402 conn(0x7fe17007adc0 0x7fe17007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 msgr2=0x7fe18419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 --2- 172.21.15.49:0/2923138303 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe184107ff0 0x7fe18419e300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009fd0 tx=0x7fe17c00b260 comp rx=0 tx=0).stop 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe182d9d640 1 -- 172.21.15.49:0/2923138303 reap_dead start 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 shutdown_connections 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 >> 172.21.15.49:0/2923138303 conn(0x7fe1840fd6d0 msgr2=0x7fe18406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 shutdown_connections 2024-09-17T11:42:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:25.498+0000 7fe1888fe640 1 -- 172.21.15.49:0/2923138303 wait complete. 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 -- 172.21.15.49:0/4034067391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8109110 msgr2=0x7f45d810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 --2- 172.21.15.49:0/4034067391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8109110 0x7f45d810f910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009be0 tx=0x7f45d002a6c0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 -- 172.21.15.49:0/4034067391 shutdown_connections 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 --2- 172.21.15.49:0/4034067391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8109110 0x7f45d810f910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 --2- 172.21.15.49:0/4034067391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8108790 0x7f45d8108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 --2- 172.21.15.49:0/4034067391 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8102d80 0x7f45d8103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 -- 172.21.15.49:0/4034067391 >> 172.21.15.49:0/4034067391 conn(0x7f45d80fe850 msgr2=0x7f45d8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.566+0000 7f45dc836640 1 -- 172.21.15.49:0/4034067391 shutdown_connections 2024-09-17T11:42:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.567+0000 7f45dc836640 1 -- 172.21.15.49:0/4034067391 wait complete. 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.567+0000 7f45dc836640 1 Processor -- start 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.567+0000 7f45dc836640 1 -- start start 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8102d80 0x7f45d8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 0x7f45d8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8109110 0x7f45d819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81124a0 con 0x7f45d8108790 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8112320 con 0x7f45d8102d80 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45dc836640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f45d8112620 con 0x7f45d8109110 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 0x7f45d8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 0x7f45d8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51974/0 (socket says 172.21.15.49:51974) 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 -- 172.21.15.49:0/3100525798 learned_addr learned my addr 172.21.15.49:0/3100525798 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45d6d9d640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8109110 0x7f45d819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45d659c640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8102d80 0x7f45d8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 -- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8109110 msgr2=0x7f45d819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8109110 0x7f45d819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 -- 172.21.15.49:0/3100525798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8102d80 msgr2=0x7f45d8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8102d80 0x7f45d8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 -- 172.21.15.49:0/3100525798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d819ea00 con 0x7f45d8108790 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45d6d9d640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f45d8109110 0x7f45d819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45d659c640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45d8102d80 0x7f45d8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cffff640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 0x7f45d8198e90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9f0 tx=0x7f45c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.568+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c000cd80 con 0x7f45d8108790 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.569+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f45c00108b0 con 0x7f45d8108790 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.569+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d819ecf0 con 0x7f45d8108790 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.569+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0018870 con 0x7f45d8108790 2024-09-17T11:42:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.569+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a53b0 con 0x7f45d8108790 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.569+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45d8100a40 con 0x7f45d8108790 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.570+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 103109+0+0 (secure 0 0 0) 0x7f45c0010430 con 0x7f45d8108790 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45cf7fe640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.49:6800/1112797402 conn(0x7f45c407adc0 0x7f45c407d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45d659c640 1 -- 172.21.15.49:0/3100525798 >> v2:172.21.15.49:6800/1112797402 conn(0x7f45c407adc0 msgr2=0x7f45c407d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f45c009eae0 con 0x7f45d8108790 2024-09-17T11:42:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45d659c640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.49:6800/1112797402 conn(0x7f45c407adc0 0x7f45c407d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45cf7fe640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.129:6800/2156637849 conn(0x7f45c4084390 0x7f45c40867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 --> v2:172.21.15.129:6800/2156637849 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45c001ce70 con 0x7f45c4084390 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f45c009ee90 con 0x7f45d8108790 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.571+0000 7f45d6d9d640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.129:6800/2156637849 conn(0x7f45c4084390 0x7f45c40867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.572+0000 7f45d6d9d640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.129:6800/2156637849 conn(0x7f45c4084390 0x7f45c40867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.573+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== osd.4 v2:172.21.15.129:6800/2156637849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45c001ce70 con 0x7f45c4084390 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.588+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 --> v2:172.21.15.129:6800/2156637849 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45d8104030 con 0x7f45c4084390 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.588+0000 7f45cf7fe640 1 -- 172.21.15.49:0/3100525798 <== osd.4 v2:172.21.15.129:6800/2156637849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f45d8104030 con 0x7f45c4084390 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.588+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 >> v2:172.21.15.129:6800/2156637849 conn(0x7f45c4084390 msgr2=0x7f45c40867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.588+0000 7f45dc836640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.129:6800/2156637849 conn(0x7f45c4084390 0x7f45c40867f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 >> v2:172.21.15.49:6800/1112797402 conn(0x7f45c407adc0 msgr2=0x7f45c407d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 --2- 172.21.15.49:0/3100525798 >> v2:172.21.15.49:6800/1112797402 conn(0x7f45c407adc0 0x7f45c407d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 msgr2=0x7f45d8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 --2- 172.21.15.49:0/3100525798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45d8108790 0x7f45d8198e90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9f0 tx=0x7f45c000eeb0 comp rx=0 tx=0).stop 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45d6d9d640 1 -- 172.21.15.49:0/3100525798 reap_dead start 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 shutdown_connections 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 >> 172.21.15.49:0/3100525798 conn(0x7f45d80fe850 msgr2=0x7f45d8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 shutdown_connections 2024-09-17T11:42:25.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:24.589+0000 7f45dc836640 1 -- 172.21.15.49:0/3100525798 wait complete. 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: Standby manager daemon x restarted 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: Standby manager daemon x started 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:42:26.105 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[40223]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: Standby manager daemon x restarted 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: Standby manager daemon x started 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:42:26.106 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:25 smithi049 ceph-mon[30017]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:42:26.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 -- 172.21.15.49:0/4101665005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938101860 msgr2=0x7f6938101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:26.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 --2- 172.21.15.49:0/4101665005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938101860 0x7f6938101c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6928009900 tx=0x7f692802a480 comp rx=0 tx=0).stop 2024-09-17T11:42:26.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 -- 172.21.15.49:0/4101665005 shutdown_connections 2024-09-17T11:42:26.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 --2- 172.21.15.49:0/4101665005 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f6938111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 --2- 172.21.15.49:0/4101665005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6938102180 0x7f693810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 --2- 172.21.15.49:0/4101665005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938101860 0x7f6938101c40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 -- 172.21.15.49:0/4101665005 >> 172.21.15.49:0/4101665005 conn(0x7f69380fd6d0 msgr2=0x7f69380ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:26.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 -- 172.21.15.49:0/4101665005 shutdown_connections 2024-09-17T11:42:26.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.121+0000 7f693d719640 1 -- 172.21.15.49:0/4101665005 wait complete. 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 Processor -- start 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 -- start start 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6938101860 0x7f693819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938102180 0x7f693819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f69381a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f69381146b0 con 0x7f6938102180 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6938114530 con 0x7f6938101860 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f693d719640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f6938114830 con 0x7f693810f410 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f69377fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f69381a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f69377fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f69381a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:52592/0 (socket says 172.21.15.49:52592) 2024-09-17T11:42:26.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f69367fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938102180 0x7f693819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.122+0000 7f69377fe640 1 -- 172.21.15.49:0/2948392622 learned_addr learned my addr 172.21.15.49:0/2948392622 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f6936ffd640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6938101860 0x7f693819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 -- 172.21.15.49:0/2948392622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6938101860 msgr2=0x7f693819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6938101860 0x7f693819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 -- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938102180 msgr2=0x7f693819b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938102180 0x7f693819b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 -- 172.21.15.49:0/2948392622 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69381a0cb0 con 0x7f693810f410 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69367fc640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6938102180 0x7f693819b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f69377fe640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f69381a04e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6930009c90 tx=0x7f693000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930010630 con 0x7f693810f410 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69381a0fa0 con 0x7f693810f410 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f6930010c50 con 0x7f693810f410 2024-09-17T11:42:26.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.123+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017b60 con 0x7f693810f410 2024-09-17T11:42:26.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.124+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69381a7660 con 0x7f693810f410 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.125+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6938063860 con 0x7f693810f410 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 18) ==== 103109+0+0 (secure 0 0 0) 0x7f6930021430 con 0x7f693810f410 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.49:6800/1112797402 conn(0x7f692007adc0 0x7f692007d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f6936ffd640 1 -- 172.21.15.49:0/2948392622 >> v2:172.21.15.49:6800/1112797402 conn(0x7f692007adc0 msgr2=0x7f692007d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/1112797402 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5942+0+0 (secure 0 0 0) 0x7f693009e640 con 0x7f693810f410 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f6936ffd640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.49:6800/1112797402 conn(0x7f692007adc0 0x7f692007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.129:6808/3044459329 conn(0x7f6920084390 0x7f69200867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 --> v2:172.21.15.129:6808/3044459329 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6920086e80 con 0x7f6920084390 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.126+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f69300a5050 con 0x7f693810f410 2024-09-17T11:42:26.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.127+0000 7f69367fc640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.129:6808/3044459329 conn(0x7f6920084390 0x7f69200867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.127+0000 7f69367fc640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.129:6808/3044459329 conn(0x7f6920084390 0x7f69200867f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:26.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.128+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== osd.6 v2:172.21.15.129:6808/3044459329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6920086e80 con 0x7f6920084390 2024-09-17T11:42:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.142+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 --> v2:172.21.15.129:6808/3044459329 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6938063c60 con 0x7f6920084390 2024-09-17T11:42:26.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.142+0000 7f691ffff640 1 -- 172.21.15.49:0/2948392622 <== osd.6 v2:172.21.15.129:6808/3044459329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6938063c60 con 0x7f6920084390 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.142+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 >> v2:172.21.15.129:6808/3044459329 conn(0x7f6920084390 msgr2=0x7f69200867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.142+0000 7f693d719640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.129:6808/3044459329 conn(0x7f6920084390 0x7f69200867f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 >> v2:172.21.15.49:6800/1112797402 conn(0x7f692007adc0 msgr2=0x7f692007d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 --2- 172.21.15.49:0/2948392622 >> v2:172.21.15.49:6800/1112797402 conn(0x7f692007adc0 0x7f692007d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 msgr2=0x7f69381a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 --2- 172.21.15.49:0/2948392622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f693810f410 0x7f69381a04e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6930009c90 tx=0x7f693000c6a0 comp rx=0 tx=0).stop 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f69377fe640 1 -- 172.21.15.49:0/2948392622 reap_dead start 2024-09-17T11:42:26.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 shutdown_connections 2024-09-17T11:42:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 >> 172.21.15.49:0/2948392622 conn(0x7f69380fd6d0 msgr2=0x7f69380ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 shutdown_connections 2024-09-17T11:42:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:26.143+0000 7f693d719640 1 -- 172.21.15.49:0/2948392622 wait complete. 2024-09-17T11:42:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: Standby manager daemon x restarted 2024-09-17T11:42:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: Standby manager daemon x started 2024-09-17T11:42:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-mon[33364]: from='mgr.? 172.21.15.129:0/1908967521' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: [17/Sep/2024:11:42:25] ENGINE Bus STARTING 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: CherryPy Checker: 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: The Application mounted at '' has an empty config. 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: [17/Sep/2024:11:42:26] ENGINE Serving on http://:::9283 2024-09-17T11:42:26.427 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x[35028]: [17/Sep/2024:11:42:26] ENGINE Bus STARTED 2024-09-17T11:42:27.015 INFO:teuthology.orchestra.run.smithi049.stdout:158913789971 2024-09-17T11:42:27.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: Active manager daemon y restarted 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: Activating manager daemon y 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: mgrmap e19: y(active, starting, since 0.0101607s), standbys: x 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:42:27.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:42:27.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: Manager daemon y is now available 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-17T11:42:27.244 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: Active manager daemon y restarted 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: Activating manager daemon y 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: mgrmap e19: y(active, starting, since 0.0101607s), standbys: x 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:42:27.245 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: Manager daemon y is now available 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.246 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: Active manager daemon y restarted 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: Activating manager daemon y 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: mgrmap e19: y(active, starting, since 0.0101607s), standbys: x 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-17T11:42:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: Manager daemon y is now available 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:27.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-17T11:42:27.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-17T11:42:27.569 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:27] ENGINE Bus STARTING 2024-09-17T11:42:27.570 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: CherryPy Checker: 2024-09-17T11:42:27.570 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: The Application mounted at '' has an empty config. 2024-09-17T11:42:27.570 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:42:27.570 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:27] ENGINE Serving on http://:::9283 2024-09-17T11:42:27.744 INFO:teuthology.orchestra.run.smithi049.stdout:55834574887 2024-09-17T11:42:27.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T11:42:27.861 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:42:27.867 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:27] ENGINE Bus STARTED 2024-09-17T11:42:27.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 -- 172.21.15.49:0/883628460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c64102dc0 msgr2=0x7f4c641031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/883628460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c64102dc0 0x7f4c641031a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400fb10 tx=0x7f4c5402e570 comp rx=0 tx=0).stop 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 -- 172.21.15.49:0/883628460 shutdown_connections 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/883628460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c64109150 0x7f4c6410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/883628460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c641087d0 0x7f4c64108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/883628460 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c64102dc0 0x7f4c641031a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 -- 172.21.15.49:0/883628460 >> 172.21.15.49:0/883628460 conn(0x7f4c640fe810 msgr2=0x7f4c64100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 -- 172.21.15.49:0/883628460 shutdown_connections 2024-09-17T11:42:27.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.879+0000 7f4c69d2b640 1 -- 172.21.15.49:0/883628460 wait complete. 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.880+0000 7f4c69d2b640 1 Processor -- start 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.880+0000 7f4c69d2b640 1 -- start start 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c64102dc0 0x7f4c641967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 0x7f4c64196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c64109150 0x7f4c6419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640772d0 con 0x7f4c64102dc0 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64077150 con 0x7f4c64109150 2024-09-17T11:42:27.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c69d2b640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f4c641105d0 con 0x7f4c641087d0 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c62ffd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 0x7f4c64196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c62ffd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 0x7f4c64196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:52684/0 (socket says 172.21.15.49:52684) 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c62ffd640 1 -- 172.21.15.49:0/1375520236 learned_addr learned my addr 172.21.15.49:0/1375520236 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c637fe640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c64102dc0 0x7f4c641967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c64109150 0x7f4c6419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 -- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 msgr2=0x7f4c64196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 0x7f4c64196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 -- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c64102dc0 msgr2=0x7f4c641967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c64102dc0 0x7f4c641967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.881+0000 7f4c63fff640 1 -- 172.21.15.49:0/1375520236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6419c890 con 0x7f4c64109150 2024-09-17T11:42:27.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c62ffd640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f4c641087d0 0x7f4c64196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c63fff640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c64109150 0x7f4c6419c0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009fd0 tx=0x7f4c5c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00ce20 con 0x7f4c64109150 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f4c5c005020 con 0x7f4c64109150 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c010690 con 0x7f4c64109150 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6419cb80 con 0x7f4c64109150 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.882+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a3240 con 0x7f4c64109150 2024-09-17T11:42:27.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.883+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c64063860 con 0x7f4c64109150 2024-09-17T11:42:27.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.885+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 102853+0+0 (secure 0 0 0) 0x7f4c5c004a40 con 0x7f4c64109150 2024-09-17T11:42:27.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.885+0000 7f4c60ff9640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6800/1163045599 conn(0x7f4c4c07aad0 0x7f4c4c07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:27.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7f4c5c09e070 con 0x7f4c64109150 2024-09-17T11:42:27.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c637fe640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6800/1163045599 conn(0x7f4c4c07aad0 0x7f4c4c07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:27.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c60ff9640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6801/501081646 conn(0x7f4c4c0843f0 0x7f4c4c086830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:27.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 --> v2:172.21.15.49:6801/501081646 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c5c0047f0 con 0x7f4c4c0843f0 2024-09-17T11:42:27.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c0a5050 con 0x7f4c64109150 2024-09-17T11:42:27.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c62ffd640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6801/501081646 conn(0x7f4c4c0843f0 0x7f4c4c086830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:27.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c62ffd640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6801/501081646 conn(0x7f4c4c0843f0 0x7f4c4c086830 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:27.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.886+0000 7f4c637fe640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6800/1163045599 conn(0x7f4c4c07aad0 0x7f4c4c07cf90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009230 tx=0x7f4c54000bc0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:27.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.888+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== osd.0 v2:172.21.15.49:6801/501081646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c5c0047f0 con 0x7f4c4c0843f0 2024-09-17T11:42:27.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.918+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 --> v2:172.21.15.49:6801/501081646 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c64063c60 con 0x7f4c4c0843f0 2024-09-17T11:42:27.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.918+0000 7f4c60ff9640 1 -- 172.21.15.49:0/1375520236 <== osd.0 v2:172.21.15.49:6801/501081646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4c64063c60 con 0x7f4c4c0843f0 2024-09-17T11:42:27.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6801/501081646 conn(0x7f4c4c0843f0 msgr2=0x7f4c4c086830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6801/501081646 conn(0x7f4c4c0843f0 0x7f4c4c086830 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6800/1163045599 conn(0x7f4c4c07aad0 msgr2=0x7f4c4c07cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/1375520236 >> v2:172.21.15.49:6800/1163045599 conn(0x7f4c4c07aad0 0x7f4c4c07cf90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009230 tx=0x7f4c54000bc0 comp rx=0 tx=0).stop 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c64109150 msgr2=0x7f4c6419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 --2- 172.21.15.49:0/1375520236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c64109150 0x7f4c6419c0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009fd0 tx=0x7f4c5c00ef10 comp rx=0 tx=0).stop 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c63fff640 1 -- 172.21.15.49:0/1375520236 reap_dead start 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 shutdown_connections 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.919+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 >> 172.21.15.49:0/1375520236 conn(0x7f4c640fe810 msgr2=0x7f4c640ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:27.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.920+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 shutdown_connections 2024-09-17T11:42:27.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:27.920+0000 7f4c69d2b640 1 -- 172.21.15.49:0/1375520236 wait complete. 2024-09-17T11:42:28.381 INFO:teuthology.orchestra.run.smithi049.stdout:210453397512 2024-09-17T11:42:28.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-17T11:42:28.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[30017]: mgrmap e20: y(active, since 1.01401s), standbys: x 2024-09-17T11:42:28.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:28.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:28.474 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[30017]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:28.475 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[40223]: mgrmap e20: y(active, since 1.01401s), standbys: x 2024-09-17T11:42:28.475 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:28.475 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:28.475 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-mon[40223]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:42:28.614+0000 7fb9a2a05640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:11:42:28] ENGINE Error in HTTPServer.serve 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: Traceback (most recent call last): 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._connections.run(self.expiration_interval) 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._run(expiration_interval) 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:42:28.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: s = self.context.wrap_socket( 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: return self.sslsocket_class._create( 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self.do_handshake() 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: self._sslobj.do_handshake() 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:42:28.728 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:28 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T11:42:28.761 INFO:teuthology.orchestra.run.smithi049.stdout:133143986201 2024-09-17T11:42:28.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-17T11:42:28.840 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:28 smithi129 ceph-mon[33364]: mgrmap e20: y(active, since 1.01401s), standbys: x 2024-09-17T11:42:28.840 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:28 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:28.840 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:28.841 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:28 smithi129 ceph-mon[33364]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:29.340 INFO:teuthology.orchestra.run.smithi049.stdout:184683593742 2024-09-17T11:42:29.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-17T11:42:29.597 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:29.597 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: [17/Sep/2024:11:42:28] ENGINE Bus STARTING 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: [17/Sep/2024:11:42:28] ENGINE Serving on http://172.21.15.49:8765 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: [17/Sep/2024:11:42:28] ENGINE Serving on https://172.21.15.49:7150 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: [17/Sep/2024:11:42:28] ENGINE Bus STARTED 2024-09-17T11:42:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[40223]: [17/Sep/2024:11:42:28] ENGINE Error in HTTPServer.serve 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: Traceback (most recent call last): 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: self._connections.run(self.expiration_interval) 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: self._run(expiration_interval) 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: s = self.context.wrap_socket( 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: return self.sslsocket_class._create( 2024-09-17T11:42:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.c.smithi049.stdout: self.do_handshake() 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.c.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.c.smithi049.stdout: self._sslobj.do_handshake() 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.c.smithi049.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: [17/Sep/2024:11:42:28] ENGINE Bus STARTING 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: [17/Sep/2024:11:42:28] ENGINE Serving on http://172.21.15.49:8765 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: [17/Sep/2024:11:42:28] ENGINE Serving on https://172.21.15.49:7150 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: [17/Sep/2024:11:42:28] ENGINE Bus STARTED 2024-09-17T11:42:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:29 smithi049 ceph-mon[30017]: [17/Sep/2024:11:42:28] ENGINE Error in HTTPServer.serve 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: Traceback (most recent call last): 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._connections.run(self.expiration_interval) 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._run(expiration_interval) 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: s = self.context.wrap_socket( 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:42:29.653 INFO:journalctl@ceph.mon.a.smithi049.stdout: return self.sslsocket_class._create( 2024-09-17T11:42:29.654 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:42:29.654 INFO:journalctl@ceph.mon.a.smithi049.stdout: self.do_handshake() 2024-09-17T11:42:29.654 INFO:journalctl@ceph.mon.a.smithi049.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:42:29.654 INFO:journalctl@ceph.mon.a.smithi049.stdout: self._sslobj.do_handshake() 2024-09-17T11:42:29.654 INFO:journalctl@ceph.mon.a.smithi049.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: [17/Sep/2024:11:42:28] ENGINE Bus STARTING 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: [17/Sep/2024:11:42:28] ENGINE Serving on http://172.21.15.49:8765 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: [17/Sep/2024:11:42:28] ENGINE Serving on https://172.21.15.49:7150 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: [17/Sep/2024:11:42:28] ENGINE Bus STARTED 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:29 smithi129 ceph-mon[33364]: [17/Sep/2024:11:42:28] ENGINE Error in HTTPServer.serve 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout: Traceback (most recent call last): 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout: self._connections.run(self.expiration_interval) 2024-09-17T11:42:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: self._run(expiration_interval) 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: s = self.context.wrap_socket( 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: return self.sslsocket_class._create( 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: self.do_handshake() 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: self._sslobj.do_handshake() 2024-09-17T11:42:29.813 INFO:journalctl@ceph.mon.b.smithi129.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T11:42:29.953 INFO:teuthology.orchestra.run.smithi049.stdout:34359738415 2024-09-17T11:42:29.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T11:42:29.966 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 -- 172.21.15.49:0/3556473495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107ff0 msgr2=0x7f6a4410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 --2- 172.21.15.49:0/3556473495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107ff0 0x7f6a4410e7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ca0 tx=0x7f6a3c02a8c0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 -- 172.21.15.49:0/3556473495 shutdown_connections 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 --2- 172.21.15.49:0/3556473495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107ff0 0x7f6a4410e7f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 --2- 172.21.15.49:0/3556473495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107670 0x7f6a44107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 --2- 172.21.15.49:0/3556473495 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 -- 172.21.15.49:0/3556473495 >> 172.21.15.49:0/3556473495 conn(0x7f6a440fd6d0 msgr2=0x7f6a440ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 -- 172.21.15.49:0/3556473495 shutdown_connections 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.612+0000 7f6a4b797640 1 -- 172.21.15.49:0/3556473495 wait complete. 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 Processor -- start 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 -- start start 2024-09-17T11:42:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 0x7f6a4419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111380 con 0x7f6a44107670 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111200 con 0x7f6a44107ff0 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.613+0000 7f6a4b797640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f6a44111500 con 0x7f6a44101c60 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52124/0 (socket says 172.21.15.49:52124) 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 -- 172.21.15.49:0/2196292340 learned_addr learned my addr 172.21.15.49:0/2196292340 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a49d34640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 0x7f6a4419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a49533640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 -- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 msgr2=0x7f6a44198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 -- 172.21.15.49:0/2196292340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 msgr2=0x7f6a4419e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 0x7f6a4419e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 -- 172.21.15.49:0/2196292340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4419ea10 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a49d34640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 0x7f6a4419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a49533640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a48d32640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009de0 tx=0x7f6a3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.614+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34016020 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.615+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f6a3400f450 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.615+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3400e610 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.615+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4419ed00 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.615+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441087c0 con 0x7f6a44107670 2024-09-17T11:42:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.616+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44102040 con 0x7f6a44107670 2024-09-17T11:42:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.616+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 21) ==== 103045+0+0 (secure 0 0 0) 0x7f6a34019020 con 0x7f6a44107670 2024-09-17T11:42:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.617+0000 7f6a3a7fc640 1 --2- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 0x7f6a3007d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.617+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7f6a3409dcd0 con 0x7f6a44107670 2024-09-17T11:42:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.617+0000 7f6a49533640 1 --2- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 0x7f6a3007d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.618+0000 7f6a49533640 1 --2- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 0x7f6a3007d200 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6a400095f0 tx=0x7f6a40005f90 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.622+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6a34020020 con 0x7f6a44107670 2024-09-17T11:42:30.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.818+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6a44109c00 con 0x7f6a44107670 2024-09-17T11:42:30.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.819+0000 7f6a3a7fc640 1 -- 172.21.15.49:0/2196292340 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a34069540 con 0x7f6a44107670 2024-09-17T11:42:30.821 INFO:teuthology.orchestra.run.smithi049.stdout:111669149726 2024-09-17T11:42:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 msgr2=0x7f6a3007d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 0x7f6a3007d200 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6a400095f0 tx=0x7f6a40005f90 comp rx=0 tx=0).stop 2024-09-17T11:42:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 msgr2=0x7f6a44198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009de0 tx=0x7f6a3400c6a0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 shutdown_connections 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6a3007ad40 0x7f6a3007d200 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a44107ff0 0x7f6a4419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a44107670 0x7f6a44198ea0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 --2- 172.21.15.49:0/2196292340 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6a44101c60 0x7f6a44198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 >> 172.21.15.49:0/2196292340 conn(0x7f6a440fd6d0 msgr2=0x7f6a440fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 shutdown_connections 2024-09-17T11:42:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:30.821+0000 7f6a4b797640 1 -- 172.21.15.49:0/2196292340 wait complete. 2024-09-17T11:42:31.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:31.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 3760M 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2196292340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 3760M 2024-09-17T11:42:31.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2196292340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-17T11:42:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 3760M 2024-09-17T11:42:31.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:31.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2196292340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-17T11:42:31.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149725 got 111669149726 for osd.3 2024-09-17T11:42:31.539 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 -- 172.21.15.49:0/1722957445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 msgr2=0x7ff1401180f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 --2- 172.21.15.49:0/1722957445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401180f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff138009ca0 tx=0x7ff13802a8c0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 -- 172.21.15.49:0/1722957445 shutdown_connections 2024-09-17T11:42:31.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 --2- 172.21.15.49:0/1722957445 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401180f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 --2- 172.21.15.49:0/1722957445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 --2- 172.21.15.49:0/1722957445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 0x7ff14010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 -- 172.21.15.49:0/1722957445 >> 172.21.15.49:0/1722957445 conn(0x7ff140072540 msgr2=0x7ff140074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 -- 172.21.15.49:0/1722957445 shutdown_connections 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.226+0000 7ff148262640 1 -- 172.21.15.49:0/1722957445 wait complete. 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 Processor -- start 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 -- start start 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 0x7ff1401a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff14011ac20 con 0x7ff140110f70 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff14011aaa0 con 0x7ff14010b590 2024-09-17T11:42:31.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff148262640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7ff14011ada0 con 0x7ff1401118f0 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stdout:184683593743 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff1457fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff1467ff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff1457fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52148/0 (socket says 172.21.15.49:52148) 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff1467ff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:52726/0 (socket says 172.21.15.49:52726) 2024-09-17T11:42:31.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff1457fd640 1 -- 172.21.15.49:0/2490803872 learned_addr learned my addr 172.21.15.49:0/2490803872 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.227+0000 7ff145ffe640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 0x7ff1401a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 -- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 msgr2=0x7ff1401a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 -- 172.21.15.49:0/2490803872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 msgr2=0x7ff1401a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 0x7ff1401a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 -- 172.21.15.49:0/2490803872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401a7230 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1467ff640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff1457fd640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff13000ea60 tx=0x7ff13000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13000cde0 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1401a7520 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff1300035f0 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1300044a0 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.228+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401adbe0 con 0x7ff140110f70 2024-09-17T11:42:31.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.229+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14010b970 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.230+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7ff130004000 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.230+0000 7ff136ffd640 1 --2- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.231+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7ff13009e1f0 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.231+0000 7ff145ffe640 1 --2- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.231+0000 7ff145ffe640 1 --2- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00bbe0 tx=0x7ff13c00b390 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.235+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff130069980 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.424+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff1401adea0 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.424+0000 7ff136ffd640 1 -- 172.21.15.49:0/2490803872 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff13006e830 con 0x7ff140110f70 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.426+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 msgr2=0x7ff12c07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.426+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00bbe0 tx=0x7ff13c00b390 comp rx=0 tx=0).stop 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.426+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 msgr2=0x7ff1401a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.426+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff13000ea60 tx=0x7ff13000ef20 comp rx=0 tx=0).stop 2024-09-17T11:42:32.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 shutdown_connections 2024-09-17T11:42:32.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:32.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1401118f0 0x7ff1401a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:32.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff140110f70 0x7ff1401a16c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:32.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 --2- 172.21.15.49:0/2490803872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14010b590 0x7ff1401a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:32.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 >> 172.21.15.49:0/2490803872 conn(0x7ff140072540 msgr2=0x7ff14010a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:32.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 shutdown_connections 2024-09-17T11:42:32.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.427+0000 7ff148262640 1 -- 172.21.15.49:0/2490803872 wait complete. 2024-09-17T11:42:32.258 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[40223]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T11:42:32.258 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2490803872' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-17T11:42:32.258 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3667354832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-17T11:42:32.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[30017]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T11:42:32.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2490803872' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-17T11:42:32.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3667354832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-17T11:42:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:32 smithi129 ceph-mon[33364]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T11:42:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2490803872' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-17T11:42:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3667354832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-17T11:42:32.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593742 got 184683593743 for osd.6 2024-09-17T11:42:32.776 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.734+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/2766902548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec101c60 msgr2=0x7fc0ec102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/2766902548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec101c60 0x7fc0ec102040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009be0 tx=0x7fc0e402a6c0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/2766902548 shutdown_connections 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/2766902548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107ff0 0x7fc0ec10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/2766902548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec107670 0x7fc0ec107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/2766902548 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec101c60 0x7fc0ec102040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/2766902548 >> 172.21.15.49:0/2766902548 conn(0x7fc0ec0fd6d0 msgr2=0x7fc0ec0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/2766902548 shutdown_connections 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.735+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/2766902548 wait complete. 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 Processor -- start 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 -- start start 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107670 0x7fc0ec198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 0x7fc0ec19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec111380 con 0x7fc0ec101c60 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec111200 con 0x7fc0ec107670 2024-09-17T11:42:33.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0f0ad5640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec111500 con 0x7fc0ec107ff0 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0ead9d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 0x7fc0ec19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52178/0 (socket says 172.21.15.49:52178) 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0ea59c640 1 -- 172.21.15.49:0/3667354832 learned_addr learned my addr 172.21.15.49:0/3667354832 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.736+0000 7fc0e9d9b640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107670 0x7fc0ec198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stdout:77309411363 2024-09-17T11:42:33.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 -- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 msgr2=0x7fc0ec19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 0x7fc0ec19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 -- 172.21.15.49:0/3667354832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107670 msgr2=0x7fc0ec198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107670 0x7fc0ec198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 -- 172.21.15.49:0/3667354832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec19e9e0 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ead9d640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 0x7fc0ec19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0ea59c640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004af0 tx=0x7fc0e4030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4041020 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec19ec70 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fc0e4003970 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4036040 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.737+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec1a5390 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.738+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec102040 con 0x7fc0ec101c60 2024-09-17T11:42:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.739+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fc0e4044020 con 0x7fc0ec101c60 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.740+0000 7fc0e37fe640 1 --2- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 0x7fc0d807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.740+0000 7fc0e9d9b640 1 --2- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 0x7fc0d807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.740+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7fc0e40bec40 con 0x7fc0ec101c60 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.740+0000 7fc0e9d9b640 1 --2- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 0x7fc0d807d290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec107340 tx=0x7fc0d4009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.744+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc0e404b020 con 0x7fc0ec101c60 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.939+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc0ec10ade0 con 0x7fc0ec101c60 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.939+0000 7fc0e37fe640 1 -- 172.21.15.49:0/3667354832 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0e408a3d0 con 0x7fc0ec101c60 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 msgr2=0x7fc0d807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 0x7fc0d807d290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec107340 tx=0x7fc0d4009260 comp rx=0 tx=0).stop 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 msgr2=0x7fc0ec198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004af0 tx=0x7fc0e4030650 comp rx=0 tx=0).stop 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 shutdown_connections 2024-09-17T11:42:33.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.941+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc0d807add0 0x7fc0d807d290 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc0ec107ff0 0x7fc0ec19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0ec107670 0x7fc0ec198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 --2- 172.21.15.49:0/3667354832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ec101c60 0x7fc0ec198930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 >> 172.21.15.49:0/3667354832 conn(0x7fc0ec0fd6d0 msgr2=0x7fc0ec06ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 shutdown_connections 2024-09-17T11:42:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:31.942+0000 7fc0f0ad5640 1 -- 172.21.15.49:0/3667354832 wait complete. 2024-09-17T11:42:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:33 smithi049 ceph-mon[40223]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:33.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:33 smithi049 ceph-mon[30017]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:33 smithi129 ceph-mon[33364]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:42:33.775 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:33.810 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:33.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411362 got 77309411363 for osd.2 2024-09-17T11:42:33.821 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:33.825 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:34.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- 172.21.15.49:0/613482289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe81090f0 msgr2=0x7febe810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 --2- 172.21.15.49:0/613482289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe81090f0 0x7febe810f8f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7febe0009860 tx=0x7febe002a340 comp rx=0 tx=0).stop 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- 172.21.15.49:0/613482289 shutdown_connections 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 --2- 172.21.15.49:0/613482289 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe81090f0 0x7febe810f8f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 --2- 172.21.15.49:0/613482289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 0x7febe8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 --2- 172.21.15.49:0/613482289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe8102d60 0x7febe8103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- 172.21.15.49:0/613482289 >> 172.21.15.49:0/613482289 conn(0x7febe80fe810 msgr2=0x7febe8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- 172.21.15.49:0/613482289 shutdown_connections 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- 172.21.15.49:0/613482289 wait complete. 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 Processor -- start 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.463+0000 7febf0432640 1 -- start start 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 0x7febe8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 0x7febe819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7febe8112480 con 0x7febe81090f0 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febe8112300 con 0x7febe8108770 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7febe8112600 con 0x7febe8102d60 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee1ce640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 0x7febe819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee1ce640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39834/0 (socket says 172.21.15.49:39834) 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee1ce640 1 -- 172.21.15.49:0/706807622 learned_addr learned my addr 172.21.15.49:0/706807622 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febed9cd640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 0x7febe8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 -- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 msgr2=0x7febe8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 -- 172.21.15.49:0/706807622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 msgr2=0x7febe8198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 0x7febe8198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 -- 172.21.15.49:0/706807622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe819ea40 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee1ce640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febee9cf640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 0x7febe819e270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febe002a860 tx=0x7febe0033d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe003f020 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe819ecd0 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7febe00044c0 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0037040 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.464+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a53f0 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.465+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe8103140 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.466+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7febe0042020 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.467+0000 7febdb7fe640 1 --2- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 0x7febd407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.467+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7febe00bde90 con 0x7febe81090f0 2024-09-17T11:42:34.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.467+0000 7febee1ce640 1 --2- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 0x7febd407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.467+0000 7febee1ce640 1 --2- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 0x7febd407d290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e8b0 tx=0x7febdc0095c0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:34.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.471+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7febe0049020 con 0x7febe81090f0 2024-09-17T11:42:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.660+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7febe810aed0 con 0x7febe81090f0 2024-09-17T11:42:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.661+0000 7febdb7fe640 1 -- 172.21.15.49:0/706807622 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7febe0089620 con 0x7febe81090f0 2024-09-17T11:42:34.663 INFO:teuthology.orchestra.run.smithi049.stdout:158913789973 2024-09-17T11:42:34.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 msgr2=0x7febd407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 0x7febd407d290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e8b0 tx=0x7febdc0095c0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 msgr2=0x7febe819e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 0x7febe819e270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febe002a860 tx=0x7febe0033d30 comp rx=0 tx=0).stop 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 shutdown_connections 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> v2:172.21.15.49:6800/1163045599 conn(0x7febd407add0 0x7febd407d290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7febe81090f0 0x7febe819e270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febe8108770 0x7febe8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 --2- 172.21.15.49:0/706807622 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7febe8102d60 0x7febe8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 >> 172.21.15.49:0/706807622 conn(0x7febe80fe810 msgr2=0x7febe8106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 shutdown_connections 2024-09-17T11:42:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.663+0000 7febf0432640 1 -- 172.21.15.49:0/706807622 wait complete. 2024-09-17T11:42:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- 172.21.15.49:0/915834055 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc810d110 msgr2=0x7fbdc810f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/915834055 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc810d110 0x7fbdc810f570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009860 tx=0x7fbdc002a340 comp rx=0 tx=0).stop 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- 172.21.15.49:0/915834055 shutdown_connections 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/915834055 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc810d110 0x7fbdc810f570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/915834055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/915834055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdc8103cb0 0x7fbdc8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- 172.21.15.49:0/915834055 >> 172.21.15.49:0/915834055 conn(0x7fbdc80fd710 msgr2=0x7fbdc80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- 172.21.15.49:0/915834055 shutdown_connections 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- 172.21.15.49:0/915834055 wait complete. 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 Processor -- start 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 -- start start 2024-09-17T11:42:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.736+0000 7fbdcec9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcec9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdc810d110 0x7fbdc819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcec9b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112490 con 0x7fbdc81045d0 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcec9b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112310 con 0x7fbdc810d110 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcec9b640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc8112610 con 0x7fbdc8103cb0 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39854/0 (socket says 172.21.15.49:39854) 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 -- 172.21.15.49:0/1967570758 learned_addr learned my addr 172.21.15.49:0/1967570758 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 -- 172.21.15.49:0/1967570758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdc810d110 msgr2=0x7fbdc819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdc810d110 0x7fbdc819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 -- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 msgr2=0x7fbdc8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 -- 172.21.15.49:0/1967570758 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc819ea60 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdc7fff640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcca37640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b4a0 tx=0x7fbdb000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0014070 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fbdb0010a90 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc819ed50 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.737+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0017b80 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.738+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a5490 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.738+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc8104090 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.739+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fbdb0017070 con 0x7fbdc8103cb0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.740+0000 7fbdc5ffb640 1 --2- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 0x7fbdb407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.740+0000 7fbdc7fff640 1 --2- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 0x7fbdb407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.740+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7fbdb009d4f0 con 0x7fbdc8103cb0 2024-09-17T11:42:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.740+0000 7fbdc7fff640 1 --2- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 0x7fbdb407d290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009a60 tx=0x7fbdb8010040 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.744+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fbdb0068d00 con 0x7fbdc8103cb0 2024-09-17T11:42:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.939+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbdc8107840 con 0x7fbdc8103cb0 2024-09-17T11:42:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.940+0000 7fbdc5ffb640 1 -- 172.21.15.49:0/1967570758 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbdb006dbb0 con 0x7fbdc8103cb0 2024-09-17T11:42:34.942 INFO:teuthology.orchestra.run.smithi049.stdout:34359738416 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 msgr2=0x7fbdb407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 0x7fbdb407d290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009a60 tx=0x7fbdb8010040 comp rx=0 tx=0).stop 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 msgr2=0x7fbdc81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b4a0 tx=0x7fbdb000b960 comp rx=0 tx=0).stop 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 shutdown_connections 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> v2:172.21.15.49:6800/1163045599 conn(0x7fbdb407add0 0x7fbdb407d290 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdc810d110 0x7fbdc819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc81045d0 0x7fbdc8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 --2- 172.21.15.49:0/1967570758 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fbdc8103cb0 0x7fbdc81989b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 >> 172.21.15.49:0/1967570758 conn(0x7fbdc80fd710 msgr2=0x7fbdc8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 shutdown_connections 2024-09-17T11:42:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:34.942+0000 7fbdcec9b640 1 -- 172.21.15.49:0/1967570758 wait complete. 2024-09-17T11:42:35.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738416 for osd.0 2024-09-17T11:42:35.601 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:35.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[30017]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T11:42:35.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/706807622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-17T11:42:35.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1967570758' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-17T11:42:35.774 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[40223]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T11:42:35.774 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/706807622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-17T11:42:35.774 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1967570758' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-17T11:42:35.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:35 smithi129 ceph-mon[33364]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T11:42:35.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/706807622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-17T11:42:35.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1967570758' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-17T11:42:36.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 -- 172.21.15.49:0/4007271676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 msgr2=0x7ffb041117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 --2- 172.21.15.49:0/4007271676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041117e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009be0 tx=0x7ffafc02a6c0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 -- 172.21.15.49:0/4007271676 shutdown_connections 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 --2- 172.21.15.49:0/4007271676 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041117e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 --2- 172.21.15.49:0/4007271676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04102160 0x7ffb0410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 --2- 172.21.15.49:0/4007271676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04101840 0x7ffb04101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 -- 172.21.15.49:0/4007271676 >> 172.21.15.49:0/4007271676 conn(0x7ffb040fd710 msgr2=0x7ffb040ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 -- 172.21.15.49:0/4007271676 shutdown_connections 2024-09-17T11:42:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.350+0000 7ffb0b771640 1 -- 172.21.15.49:0/4007271676 wait complete. 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 Processor -- start 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 -- start start 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04101840 0x7ffb0419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 0x7ffb0419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffb04114680 con 0x7ffb04102160 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb04114500 con 0x7ffb04101840 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0b771640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7ffb04114800 con 0x7ffb0410f3f0 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb08d0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 0x7ffb0419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39870/0 (socket says 172.21.15.49:39870) 2024-09-17T11:42:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 -- 172.21.15.49:0/1192609888 learned_addr learned my addr 172.21.15.49:0/1192609888 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb0950d640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04101840 0x7ffb0419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 -- 172.21.15.49:0/1192609888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04101840 msgr2=0x7ffb0419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04101840 0x7ffb0419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 -- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 msgr2=0x7ffb0419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 0x7ffb0419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb09d0e640 1 -- 172.21.15.49:0/1192609888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb041a0c70 con 0x7ffb0410f3f0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.351+0000 7ffb08d0c640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 0x7ffb0419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffb09d0e640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffafc02abe0 tx=0x7ffafc0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc041020 con 0x7ffb0410f3f0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ffafc030740 con 0x7ffb0410f3f0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb041a0f00 con 0x7ffb0410f3f0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb041a7620 con 0x7ffb0410f3f0 2024-09-17T11:42:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.352+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc036040 con 0x7ffb0410f3f0 2024-09-17T11:42:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.353+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb04101c20 con 0x7ffb0410f3f0 2024-09-17T11:42:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.353+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7ffafc044020 con 0x7ffb0410f3f0 2024-09-17T11:42:36.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.354+0000 7ffafa7fc640 1 --2- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 0x7ffaf007d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.354+0000 7ffb0950d640 1 --2- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 0x7ffaf007d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.354+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7ffafc0bea70 con 0x7ffb0410f3f0 2024-09-17T11:42:36.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.355+0000 7ffb0950d640 1 --2- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 0x7ffaf007d290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009530 tx=0x7ffb00009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:36.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.359+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ffafc04b020 con 0x7ffb0410f3f0 2024-09-17T11:42:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.534+0000 7ff1488d4640 1 -- 172.21.15.49:0/2088508913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 msgr2=0x7ff144101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.534+0000 7ff1488d4640 1 --2- 172.21.15.49:0/2088508913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff144101c60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff134009900 tx=0x7ff13402a480 comp rx=0 tx=0).stop 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 -- 172.21.15.49:0/2088508913 shutdown_connections 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 --2- 172.21.15.49:0/2088508913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff144111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 --2- 172.21.15.49:0/2088508913 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 --2- 172.21.15.49:0/2088508913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff144101c60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 -- 172.21.15.49:0/2088508913 >> 172.21.15.49:0/2088508913 conn(0x7ff1440fd6d0 msgr2=0x7ff1440ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 -- 172.21.15.49:0/2088508913 shutdown_connections 2024-09-17T11:42:36.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 -- 172.21.15.49:0/2088508913 wait complete. 2024-09-17T11:42:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 Processor -- start 2024-09-17T11:42:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.535+0000 7ff1488d4640 1 -- start start 2024-09-17T11:42:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff1441a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff144114610 con 0x7ff144101880 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff144114490 con 0x7ff14410f430 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff1488d4640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7ff144114790 con 0x7ff1441021a0 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39310/0 (socket says 172.21.15.49:39310) 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 -- 172.21.15.49:0/1295690657 learned_addr learned my addr 172.21.15.49:0/1295690657 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff141d9b640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff142d9d640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff1441a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 -- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 msgr2=0x7ff14419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 -- 172.21.15.49:0/1295690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 msgr2=0x7ff1441a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff1441a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 -- 172.21.15.49:0/1295690657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1441a0ca0 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff142d9d640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff1441a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff141d9b640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.536+0000 7ff14259c640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff134005940 tx=0x7ff134032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.537+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13403f030 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.537+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff134005480 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.537+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1441a0f30 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.537+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134035070 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.537+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1441a7650 con 0x7ff144101880 2024-09-17T11:42:36.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.538+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff144101c60 con 0x7ff144101880 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.539+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7ff134042030 con 0x7ff144101880 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.539+0000 7ff1337fe640 1 --2- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.540+0000 7ff141d9b640 1 --2- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.540+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7ff1340be2f0 con 0x7ff144101880 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.540+0000 7ff141d9b640 1 --2- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff1380095f0 tx=0x7ff138005f90 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:36.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.544+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff134089b00 con 0x7ff144101880 2024-09-17T11:42:36.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.548+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffb041a7850 con 0x7ffb0410f3f0 2024-09-17T11:42:36.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.548+0000 7ffafa7fc640 1 -- 172.21.15.49:0/1192609888 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffafc08a280 con 0x7ffb0410f3f0 2024-09-17T11:42:36.550 INFO:teuthology.orchestra.run.smithi049.stdout:133143986203 2024-09-17T11:42:36.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 msgr2=0x7ffaf007d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 0x7ffaf007d290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009530 tx=0x7ffb00009260 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 msgr2=0x7ffb041a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffafc02abe0 tx=0x7ffafc0305d0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 shutdown_connections 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> v2:172.21.15.49:6800/1163045599 conn(0x7ffaf007add0 0x7ffaf007d290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ffb0410f3f0 0x7ffb041a04a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb04102160 0x7ffb0419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 --2- 172.21.15.49:0/1192609888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb04101840 0x7ffb0419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.550+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 >> 172.21.15.49:0/1192609888 conn(0x7ffb040fd710 msgr2=0x7ffb0410f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.551+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 shutdown_connections 2024-09-17T11:42:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.551+0000 7ffb0b771640 1 -- 172.21.15.49:0/1192609888 wait complete. 2024-09-17T11:42:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.732+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff1441a7940 con 0x7ff144101880 2024-09-17T11:42:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.732+0000 7ff1337fe640 1 -- 172.21.15.49:0/1295690657 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff13408e9b0 con 0x7ff144101880 2024-09-17T11:42:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:210453397514 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 msgr2=0x7ff12c07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff1380095f0 tx=0x7ff138005f90 comp rx=0 tx=0).stop 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 msgr2=0x7ff14419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff134005940 tx=0x7ff134032c70 comp rx=0 tx=0).stop 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 shutdown_connections 2024-09-17T11:42:36.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff12c07add0 0x7ff12c07d290 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff14410f430 0x7ff1441a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff1441021a0 0x7ff14419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 --2- 172.21.15.49:0/1295690657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff144101880 0x7ff14419abf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 >> 172.21.15.49:0/1295690657 conn(0x7ff1440fd6d0 msgr2=0x7ff14410fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 shutdown_connections 2024-09-17T11:42:36.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:36.735+0000 7ff1488d4640 1 -- 172.21.15.49:0/1295690657 wait complete. 2024-09-17T11:42:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:42:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:42:37.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397512 got 210453397514 for osd.7 2024-09-17T11:42:37.394 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1959652057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd810f3f0 msgr2=0x7f0cd81117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1959652057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd810f3f0 0x7f0cd81117e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009be0 tx=0x7f0cd002a6c0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1959652057 shutdown_connections 2024-09-17T11:42:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1959652057 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd810f3f0 0x7f0cd81117e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1959652057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 0x7f0cd810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1959652057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd8101840 0x7f0cd8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1959652057 >> 172.21.15.49:0/1959652057 conn(0x7f0cd80fd710 msgr2=0x7f0cd80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1959652057 shutdown_connections 2024-09-17T11:42:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.591+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1959652057 wait complete. 2024-09-17T11:42:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 Processor -- start 2024-09-17T11:42:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 -- start start 2024-09-17T11:42:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 0x7f0cd819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 0x7f0cd81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8114680 con 0x7f0cd810f3f0 2024-09-17T11:42:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8114500 con 0x7f0cd8102160 2024-09-17T11:42:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdfa39640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd8114800 con 0x7f0cd8101840 2024-09-17T11:42:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdd7d5640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cddfd6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 0x7f0cd81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdd7d5640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39922/0 (socket says 172.21.15.49:39922) 2024-09-17T11:42:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.592+0000 7f0cdd7d5640 1 -- 172.21.15.49:0/1514955675 learned_addr learned my addr 172.21.15.49:0/1514955675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cdcfd4640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 0x7f0cd819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 -- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 msgr2=0x7f0cd819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 -- 172.21.15.49:0/1514955675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 msgr2=0x7f0cd819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 0x7f0cd819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 -- 172.21.15.49:0/1514955675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81a0c20 con 0x7f0cd810f3f0 2024-09-17T11:42:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cdd7d5640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cddfd6640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 0x7f0cd81a0450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0034040 tx=0x7f0cd0030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0041020 con 0x7f0cd810f3f0 2024-09-17T11:42:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a0eb0 con 0x7f0cd810f3f0 2024-09-17T11:42:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f0cd0003970 con 0x7f0cd810f3f0 2024-09-17T11:42:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0036040 con 0x7f0cd810f3f0 2024-09-17T11:42:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.593+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a75d0 con 0x7f0cd810f3f0 2024-09-17T11:42:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.594+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8101c20 con 0x7f0cd810f3f0 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.595+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f0cd0044020 con 0x7f0cd810f3f0 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.596+0000 7f0cce7fc640 1 --2- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 0x7f0cc407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.596+0000 7f0cdd7d5640 1 --2- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 0x7f0cc407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.596+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7f0cd00beb60 con 0x7f0cd810f3f0 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.596+0000 7f0cdd7d5640 1 --2- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 0x7f0cc407d290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40096c0 tx=0x7f0cd4009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.600+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f0cd004b020 con 0x7f0cd810f3f0 2024-09-17T11:42:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[40223]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-17T11:42:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1192609888' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-17T11:42:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1295690657' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-17T11:42:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[30017]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-17T11:42:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1192609888' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-17T11:42:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1295690657' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-17T11:42:37.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.798+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0cd81a7800 con 0x7f0cd810f3f0 2024-09-17T11:42:37.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.799+0000 7f0cce7fc640 1 -- 172.21.15.49:0/1514955675 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0cd008a2f0 con 0x7f0cd810f3f0 2024-09-17T11:42:37.802 INFO:teuthology.orchestra.run.smithi049.stdout:55834574889 2024-09-17T11:42:37.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 msgr2=0x7f0cc407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:37.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 0x7f0cc407d290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40096c0 tx=0x7f0cd4009260 comp rx=0 tx=0).stop 2024-09-17T11:42:37.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 msgr2=0x7f0cd81a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:37.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 0x7f0cd81a0450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0034040 tx=0x7f0cd0030650 comp rx=0 tx=0).stop 2024-09-17T11:42:37.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 shutdown_connections 2024-09-17T11:42:37.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> v2:172.21.15.49:6800/1163045599 conn(0x7f0cc407add0 0x7f0cc407d290 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cd810f3f0 0x7f0cd81a0450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cd8102160 0x7f0cd819b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 --2- 172.21.15.49:0/1514955675 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f0cd8101840 0x7f0cd819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:37.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 >> 172.21.15.49:0/1514955675 conn(0x7f0cd80fd710 msgr2=0x7f0cd810f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:37.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 shutdown_connections 2024-09-17T11:42:37.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:37.801+0000 7f0cdfa39640 1 -- 172.21.15.49:0/1514955675 wait complete. 2024-09-17T11:42:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:37 smithi129 ceph-mon[33364]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-17T11:42:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1192609888' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-17T11:42:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1295690657' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-17T11:42:37.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:42:38.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789971 got 158913789973 for osd.5 2024-09-17T11:42:38.046 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:38.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574887 got 55834574889 for osd.1 2024-09-17T11:42:38.706 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:38.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:39.108 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:38 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.108 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:38 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.108 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1514955675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1514955675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1514955675' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-17T11:42:39.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986201 got 133143986203 for osd.4 2024-09-17T11:42:39.205 DEBUG:teuthology.parallel:result is None 2024-09-17T11:42:39.205 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T11:42:39.205 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T11:42:39.475 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:39.886 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:39.886 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.886 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 3760M 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:42:39.887 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:39.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 3760M 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:39.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 -- 172.21.15.49:0/3180399129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 msgr2=0x7f3780107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 --2- 172.21.15.49:0/3180399129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780107ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3774009900 tx=0x7f377402a410 comp rx=0 tx=0).stop 2024-09-17T11:42:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 -- 172.21.15.49:0/3180399129 shutdown_connections 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 --2- 172.21.15.49:0/3180399129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780107ff0 0x7f378010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 --2- 172.21.15.49:0/3180399129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780107ab0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 --2- 172.21.15.49:0/3180399129 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780101c60 0x7f3780102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 -- 172.21.15.49:0/3180399129 >> 172.21.15.49:0/3180399129 conn(0x7f37800fd6d0 msgr2=0x7f37800ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.968+0000 7f3786292640 1 -- 172.21.15.49:0/3180399129 shutdown_connections 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.969+0000 7f3786292640 1 -- 172.21.15.49:0/3180399129 wait complete. 2024-09-17T11:42:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.969+0000 7f3786292640 1 Processor -- start 2024-09-17T11:42:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.969+0000 7f3786292640 1 -- start start 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 0x7f3780198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f37801113c0 con 0x7f3780107670 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3780111240 con 0x7f3780101c60 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f3786292640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f3780111540 con 0x7f3780107ff0 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f377f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39930/0 (socket says 172.21.15.49:39930) 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f377f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39366/0 (socket says 172.21.15.49:39366) 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 -- 172.21.15.49:0/665788818 learned_addr learned my addr 172.21.15.49:0/665788818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f377ffff640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 0x7f3780198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 -- 172.21.15.49:0/665788818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 msgr2=0x7f3780198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 0x7f3780198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 -- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 msgr2=0x7f3780198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f378482f640 1 -- 172.21.15.49:0/665788818 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378019ea30 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f377ffff640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 0x7f3780198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.970+0000 7f377f7fe640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f378482f640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f377800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778017070 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3778003df0 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378019ed20 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778010460 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.971+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a53e0 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.972+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3780102040 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.973+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f3778003970 con 0x7f3780107ff0 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.973+0000 7f377d7fa640 1 --2- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 0x7f376807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.974+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7f377809dfe0 con 0x7f3780107ff0 2024-09-17T11:42:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.974+0000 7f377ffff640 1 --2- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 0x7f376807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.974+0000 7f377ffff640 1 --2- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 0x7f376807d290 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3770009fc0 tx=0x7f3770005f90 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:39.978+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f37780697b0 con 0x7f3780107ff0 2024-09-17T11:42:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 3760M 2024-09-17T11:42:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi049:/etc/ceph/ceph.conf 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:40.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.191+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 --> v2:172.21.15.49:6800/1163045599 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3780063590 con 0x7f376807add0 2024-09-17T11:42:40.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.197+0000 7f377d7fa640 1 -- 172.21.15.49:0/665788818 <== mgr.14619 v2:172.21.15.49:6800/1163045599 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346454 (secure 0 0 0) 0x7f3780063590 con 0x7f376807add0 2024-09-17T11:42:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:40.205 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-09-17T11:42:40.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 msgr2=0x7f376807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:40.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 0x7f376807d290 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3770009fc0 tx=0x7f3770005f90 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 msgr2=0x7f378019e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f377800b260 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 shutdown_connections 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> v2:172.21.15.49:6800/1163045599 conn(0x7f376807add0 0x7f376807d290 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3780107ff0 0x7f378019e260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3780107670 0x7f3780198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 --2- 172.21.15.49:0/665788818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3780101c60 0x7f3780198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 >> 172.21.15.49:0/665788818 conn(0x7f37800fd6d0 msgr2=0x7f378006ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 shutdown_connections 2024-09-17T11:42:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:40.201+0000 7f3786292640 1 -- 172.21.15.49:0/665788818 wait complete. 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[30017]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[30017]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[30017]: Deploying daemon alertmanager.a on smithi049 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[40223]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[40223]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:40.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:40 smithi049 ceph-mon[40223]: Deploying daemon alertmanager.a on smithi049 2024-09-17T11:42:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:40 smithi129 ceph-mon[33364]: Updating smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:40 smithi129 ceph-mon[33364]: Updating smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T11:42:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:40 smithi129 ceph-mon[33364]: Deploying daemon alertmanager.a on smithi049 2024-09-17T11:42:41.181 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-17T11:42:38.436649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":884,"num_read_kb":741,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239600,"kb_used_data":6748,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721616,"statfs":{"total":767960285184,"available":767714934784,"internally_reserved":0,"allocated":6909952,"data_stored":3694729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.991780"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426066+0000","last_change":"2024-09-17T11:41:55.989582+0000","last_active":"2024-09-17T11:42:26.426066+0000","last_peered":"2024-09-17T11:42:26.426066+0000","last_clean":"2024-09-17T11:42:26.426066+0000","last_became_active":"2024-09-17T11:41:55.989141+0000","last_became_peered":"2024-09-17T11:41:55.989141+0000","last_unstale":"2024-09-17T11:42:26.426066+0000","last_undegraded":"2024-09-17T11:42:26.426066+0000","last_fullsized":"2024-09-17T11:42:26.426066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:12.831169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813707+0000","last_change":"2024-09-17T11:41:57.991084+0000","last_active":"2024-09-17T11:42:26.813707+0000","last_peered":"2024-09-17T11:42:26.813707+0000","last_clean":"2024-09-17T11:42:26.813707+0000","last_became_active":"2024-09-17T11:41:57.990552+0000","last_became_peered":"2024-09-17T11:41:57.990552+0000","last_unstale":"2024-09-17T11:42:26.813707+0000","last_undegraded":"2024-09-17T11:42:26.813707+0000","last_fullsized":"2024-09-17T11:42:26.813707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:11.988423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813731+0000","last_change":"2024-09-17T11:41:59.993877+0000","last_active":"2024-09-17T11:42:26.813731+0000","last_peered":"2024-09-17T11:42:26.813731+0000","last_clean":"2024-09-17T11:42:26.813731+0000","last_became_active":"2024-09-17T11:41:59.993555+0000","last_became_peered":"2024-09-17T11:41:59.993555+0000","last_unstale":"2024-09-17T11:42:26.813731+0000","last_undegraded":"2024-09-17T11:42:26.813731+0000","last_fullsized":"2024-09-17T11:42:26.813731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:24.333635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814115+0000","last_change":"2024-09-17T11:42:02.003670+0000","last_active":"2024-09-17T11:42:26.814115+0000","last_peered":"2024-09-17T11:42:26.814115+0000","last_clean":"2024-09-17T11:42:26.814115+0000","last_became_active":"2024-09-17T11:42:02.002935+0000","last_became_peered":"2024-09-17T11:42:02.002935+0000","last_unstale":"2024-09-17T11:42:26.814115+0000","last_undegraded":"2024-09-17T11:42:26.814115+0000","last_fullsized":"2024-09-17T11:42:26.814115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:51.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427727+0000","last_change":"2024-09-17T11:42:02.000886+0000","last_active":"2024-09-17T11:42:26.427727+0000","last_peered":"2024-09-17T11:42:26.427727+0000","last_clean":"2024-09-17T11:42:26.427727+0000","last_became_active":"2024-09-17T11:42:01.999941+0000","last_became_peered":"2024-09-17T11:42:01.999941+0000","last_unstale":"2024-09-17T11:42:26.427727+0000","last_undegraded":"2024-09-17T11:42:26.427727+0000","last_fullsized":"2024-09-17T11:42:26.427727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:35:02.189227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429628+0000","last_change":"2024-09-17T11:41:55.974362+0000","last_active":"2024-09-17T11:42:26.429628+0000","last_peered":"2024-09-17T11:42:26.429628+0000","last_clean":"2024-09-17T11:42:26.429628+0000","last_became_active":"2024-09-17T11:41:55.974036+0000","last_became_peered":"2024-09-17T11:41:55.974036+0000","last_unstale":"2024-09-17T11:42:26.429628+0000","last_undegraded":"2024-09-17T11:42:26.429628+0000","last_fullsized":"2024-09-17T11:42:26.429628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:07:21.608904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427933+0000","last_change":"2024-09-17T11:41:57.986171+0000","last_active":"2024-09-17T11:42:26.427933+0000","last_peered":"2024-09-17T11:42:26.427933+0000","last_clean":"2024-09-17T11:42:26.427933+0000","last_became_active":"2024-09-17T11:41:57.985808+0000","last_became_peered":"2024-09-17T11:41:57.985808+0000","last_unstale":"2024-09-17T11:42:26.427933+0000","last_undegraded":"2024-09-17T11:42:26.427933+0000","last_fullsized":"2024-09-17T11:42:26.427933+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:50:58.594905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813644+0000","last_change":"2024-09-17T11:41:59.994549+0000","last_active":"2024-09-17T11:42:26.813644+0000","last_peered":"2024-09-17T11:42:26.813644+0000","last_clean":"2024-09-17T11:42:26.813644+0000","last_became_active":"2024-09-17T11:41:59.994247+0000","last_became_peered":"2024-09-17T11:41:59.994247+0000","last_unstale":"2024-09-17T11:42:26.813644+0000","last_undegraded":"2024-09-17T11:42:26.813644+0000","last_fullsized":"2024-09-17T11:42:26.813644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:29:15.122502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815219+0000","last_change":"2024-09-17T11:41:55.981740+0000","last_active":"2024-09-17T11:42:26.815219+0000","last_peered":"2024-09-17T11:42:26.815219+0000","last_clean":"2024-09-17T11:42:26.815219+0000","last_became_active":"2024-09-17T11:41:55.981484+0000","last_became_peered":"2024-09-17T11:41:55.981484+0000","last_unstale":"2024-09-17T11:42:26.815219+0000","last_undegraded":"2024-09-17T11:42:26.815219+0000","last_fullsized":"2024-09-17T11:42:26.815219+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:18.526591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814513+0000","last_change":"2024-09-17T11:41:57.988080+0000","last_active":"2024-09-17T11:42:26.814513+0000","last_peered":"2024-09-17T11:42:26.814513+0000","last_clean":"2024-09-17T11:42:26.814513+0000","last_became_active":"2024-09-17T11:41:57.987836+0000","last_became_peered":"2024-09-17T11:41:57.987836+0000","last_unstale":"2024-09-17T11:42:26.814513+0000","last_undegraded":"2024-09-17T11:42:26.814513+0000","last_fullsized":"2024-09-17T11:42:26.814513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:58:02.180071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814368+0000","last_change":"2024-09-17T11:41:59.996325+0000","last_active":"2024-09-17T11:42:26.814368+0000","last_peered":"2024-09-17T11:42:26.814368+0000","last_clean":"2024-09-17T11:42:26.814368+0000","last_became_active":"2024-09-17T11:41:59.995335+0000","last_became_peered":"2024-09-17T11:41:59.995335+0000","last_unstale":"2024-09-17T11:42:26.814368+0000","last_undegraded":"2024-09-17T11:42:26.814368+0000","last_fullsized":"2024-09-17T11:42:26.814368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:02:53.382608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424763+0000","last_change":"2024-09-17T11:42:01.998933+0000","last_active":"2024-09-17T11:42:26.424763+0000","last_peered":"2024-09-17T11:42:26.424763+0000","last_clean":"2024-09-17T11:42:26.424763+0000","last_became_active":"2024-09-17T11:42:01.998619+0000","last_became_peered":"2024-09-17T11:42:01.998619+0000","last_unstale":"2024-09-17T11:42:26.424763+0000","last_undegraded":"2024-09-17T11:42:26.424763+0000","last_fullsized":"2024-09-17T11:42:26.424763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:53.371794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814693+0000","last_change":"2024-09-17T11:41:55.982408+0000","last_active":"2024-09-17T11:42:26.814693+0000","last_peered":"2024-09-17T11:42:26.814693+0000","last_clean":"2024-09-17T11:42:26.814693+0000","last_became_active":"2024-09-17T11:41:55.982121+0000","last_became_peered":"2024-09-17T11:41:55.982121+0000","last_unstale":"2024-09-17T11:42:26.814693+0000","last_undegraded":"2024-09-17T11:42:26.814693+0000","last_fullsized":"2024-09-17T11:42:26.814693+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:34:03.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'5","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814046+0000","last_change":"2024-09-17T11:41:57.990000+0000","last_active":"2024-09-17T11:42:26.814046+0000","last_peered":"2024-09-17T11:42:26.814046+0000","last_clean":"2024-09-17T11:42:26.814046+0000","last_became_active":"2024-09-17T11:41:57.989698+0000","last_became_peered":"2024-09-17T11:41:57.989698+0000","last_unstale":"2024-09-17T11:42:26.814046+0000","last_undegraded":"2024-09-17T11:42:26.814046+0000","last_fullsized":"2024-09-17T11:42:26.814046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:48.766237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426238+0000","last_change":"2024-09-17T11:41:59.997600+0000","last_active":"2024-09-17T11:42:26.426238+0000","last_peered":"2024-09-17T11:42:26.426238+0000","last_clean":"2024-09-17T11:42:26.426238+0000","last_became_active":"2024-09-17T11:41:59.997305+0000","last_became_peered":"2024-09-17T11:41:59.997305+0000","last_unstale":"2024-09-17T11:42:26.426238+0000","last_undegraded":"2024-09-17T11:42:26.426238+0000","last_fullsized":"2024-09-17T11:42:26.426238+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:23.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814010+0000","last_change":"2024-09-17T11:42:01.997327+0000","last_active":"2024-09-17T11:42:26.814010+0000","last_peered":"2024-09-17T11:42:26.814010+0000","last_clean":"2024-09-17T11:42:26.814010+0000","last_became_active":"2024-09-17T11:42:01.996978+0000","last_became_peered":"2024-09-17T11:42:01.996978+0000","last_unstale":"2024-09-17T11:42:26.814010+0000","last_undegraded":"2024-09-17T11:42:26.814010+0000","last_fullsized":"2024-09-17T11:42:26.814010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:55:26.728237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814071+0000","last_change":"2024-09-17T11:42:02.003899+0000","last_active":"2024-09-17T11:42:26.814071+0000","last_peered":"2024-09-17T11:42:26.814071+0000","last_clean":"2024-09-17T11:42:26.814071+0000","last_became_active":"2024-09-17T11:42:02.003579+0000","last_became_peered":"2024-09-17T11:42:02.003579+0000","last_unstale":"2024-09-17T11:42:26.814071+0000","last_undegraded":"2024-09-17T11:42:26.814071+0000","last_fullsized":"2024-09-17T11:42:26.814071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:55.341047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425837+0000","last_change":"2024-09-17T11:41:55.986976+0000","last_active":"2024-09-17T11:42:26.425837+0000","last_peered":"2024-09-17T11:42:26.425837+0000","last_clean":"2024-09-17T11:42:26.425837+0000","last_became_active":"2024-09-17T11:41:55.986667+0000","last_became_peered":"2024-09-17T11:41:55.986667+0000","last_unstale":"2024-09-17T11:42:26.425837+0000","last_undegraded":"2024-09-17T11:42:26.425837+0000","last_fullsized":"2024-09-17T11:42:26.425837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:48.330375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814766+0000","last_change":"2024-09-17T11:41:57.989989+0000","last_active":"2024-09-17T11:42:26.814766+0000","last_peered":"2024-09-17T11:42:26.814766+0000","last_clean":"2024-09-17T11:42:26.814766+0000","last_became_active":"2024-09-17T11:41:57.989671+0000","last_became_peered":"2024-09-17T11:41:57.989671+0000","last_unstale":"2024-09-17T11:42:26.814766+0000","last_undegraded":"2024-09-17T11:42:26.814766+0000","last_fullsized":"2024-09-17T11:42:26.814766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.169198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814540+0000","last_change":"2024-09-17T11:41:59.993889+0000","last_active":"2024-09-17T11:42:26.814540+0000","last_peered":"2024-09-17T11:42:26.814540+0000","last_clean":"2024-09-17T11:42:26.814540+0000","last_became_active":"2024-09-17T11:41:59.993562+0000","last_became_peered":"2024-09-17T11:41:59.993562+0000","last_unstale":"2024-09-17T11:42:26.814540+0000","last_undegraded":"2024-09-17T11:42:26.814540+0000","last_fullsized":"2024-09-17T11:42:26.814540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:14:59.117798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428794+0000","last_change":"2024-09-17T11:42:02.003394+0000","last_active":"2024-09-17T11:42:26.428794+0000","last_peered":"2024-09-17T11:42:26.428794+0000","last_clean":"2024-09-17T11:42:26.428794+0000","last_became_active":"2024-09-17T11:42:02.003094+0000","last_became_peered":"2024-09-17T11:42:02.003094+0000","last_unstale":"2024-09-17T11:42:26.428794+0000","last_undegraded":"2024-09-17T11:42:26.428794+0000","last_fullsized":"2024-09-17T11:42:26.428794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:51:07.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814692+0000","last_change":"2024-09-17T11:41:55.987999+0000","last_active":"2024-09-17T11:42:26.814692+0000","last_peered":"2024-09-17T11:42:26.814692+0000","last_clean":"2024-09-17T11:42:26.814692+0000","last_became_active":"2024-09-17T11:41:55.987725+0000","last_became_peered":"2024-09-17T11:41:55.987725+0000","last_unstale":"2024-09-17T11:42:26.814692+0000","last_undegraded":"2024-09-17T11:42:26.814692+0000","last_fullsized":"2024-09-17T11:42:26.814692+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:17:10.141698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428002+0000","last_change":"2024-09-17T11:41:57.989986+0000","last_active":"2024-09-17T11:42:26.428002+0000","last_peered":"2024-09-17T11:42:26.428002+0000","last_clean":"2024-09-17T11:42:26.428002+0000","last_became_active":"2024-09-17T11:41:57.989698+0000","last_became_peered":"2024-09-17T11:41:57.989698+0000","last_unstale":"2024-09-17T11:42:26.428002+0000","last_undegraded":"2024-09-17T11:42:26.428002+0000","last_fullsized":"2024-09-17T11:42:26.428002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:11.676870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814196+0000","last_change":"2024-09-17T11:41:59.984949+0000","last_active":"2024-09-17T11:42:26.814196+0000","last_peered":"2024-09-17T11:42:26.814196+0000","last_clean":"2024-09-17T11:42:26.814196+0000","last_became_active":"2024-09-17T11:41:59.984678+0000","last_became_peered":"2024-09-17T11:41:59.984678+0000","last_unstale":"2024-09-17T11:42:26.814196+0000","last_undegraded":"2024-09-17T11:42:26.814196+0000","last_fullsized":"2024-09-17T11:42:26.814196+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:10.056996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426497+0000","last_change":"2024-09-17T11:42:02.004483+0000","last_active":"2024-09-17T11:42:26.426497+0000","last_peered":"2024-09-17T11:42:26.426497+0000","last_clean":"2024-09-17T11:42:26.426497+0000","last_became_active":"2024-09-17T11:42:02.004055+0000","last_became_peered":"2024-09-17T11:42:02.004055+0000","last_unstale":"2024-09-17T11:42:26.426497+0000","last_undegraded":"2024-09-17T11:42:26.426497+0000","last_fullsized":"2024-09-17T11:42:26.426497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:34.942282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813865+0000","last_change":"2024-09-17T11:41:55.982105+0000","last_active":"2024-09-17T11:42:26.813865+0000","last_peered":"2024-09-17T11:42:26.813865+0000","last_clean":"2024-09-17T11:42:26.813865+0000","last_became_active":"2024-09-17T11:41:55.981797+0000","last_became_peered":"2024-09-17T11:41:55.981797+0000","last_unstale":"2024-09-17T11:42:26.813865+0000","last_undegraded":"2024-09-17T11:42:26.813865+0000","last_fullsized":"2024-09-17T11:42:26.813865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:22:38.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426450+0000","last_change":"2024-09-17T11:41:58.208884+0000","last_active":"2024-09-17T11:42:26.426450+0000","last_peered":"2024-09-17T11:42:26.426450+0000","last_clean":"2024-09-17T11:42:26.426450+0000","last_became_active":"2024-09-17T11:41:58.208509+0000","last_became_peered":"2024-09-17T11:41:58.208509+0000","last_unstale":"2024-09-17T11:42:26.426450+0000","last_undegraded":"2024-09-17T11:42:26.426450+0000","last_fullsized":"2024-09-17T11:42:26.426450+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:35.337301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041127+0000","last_change":"2024-09-17T11:41:59.991917+0000","last_active":"2024-09-17T11:42:31.041127+0000","last_peered":"2024-09-17T11:42:31.041127+0000","last_clean":"2024-09-17T11:42:31.041127+0000","last_became_active":"2024-09-17T11:41:59.991342+0000","last_became_peered":"2024-09-17T11:41:59.991342+0000","last_unstale":"2024-09-17T11:42:31.041127+0000","last_undegraded":"2024-09-17T11:42:31.041127+0000","last_fullsized":"2024-09-17T11:42:31.041127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:28.808385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815106+0000","last_change":"2024-09-17T11:41:57.995355+0000","last_active":"2024-09-17T11:42:26.815106+0000","last_peered":"2024-09-17T11:42:26.815106+0000","last_clean":"2024-09-17T11:42:26.815106+0000","last_became_active":"2024-09-17T11:41:57.995045+0000","last_became_peered":"2024-09-17T11:41:57.995045+0000","last_unstale":"2024-09-17T11:42:26.815106+0000","last_undegraded":"2024-09-17T11:42:26.815106+0000","last_fullsized":"2024-09-17T11:42:26.815106+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:40:07.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429231+0000","last_change":"2024-09-17T11:41:55.983669+0000","last_active":"2024-09-17T11:42:26.429231+0000","last_peered":"2024-09-17T11:42:26.429231+0000","last_clean":"2024-09-17T11:42:26.429231+0000","last_became_active":"2024-09-17T11:41:55.983387+0000","last_became_peered":"2024-09-17T11:41:55.983387+0000","last_unstale":"2024-09-17T11:42:26.429231+0000","last_undegraded":"2024-09-17T11:42:26.429231+0000","last_fullsized":"2024-09-17T11:42:26.429231+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:24:15.164436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041485+0000","last_change":"2024-09-17T11:41:59.990356+0000","last_active":"2024-09-17T11:42:31.041485+0000","last_peered":"2024-09-17T11:42:31.041485+0000","last_clean":"2024-09-17T11:42:31.041485+0000","last_became_active":"2024-09-17T11:41:59.990042+0000","last_became_peered":"2024-09-17T11:41:59.990042+0000","last_unstale":"2024-09-17T11:42:31.041485+0000","last_undegraded":"2024-09-17T11:42:31.041485+0000","last_fullsized":"2024-09-17T11:42:31.041485+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:15:45.670580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814752+0000","last_change":"2024-09-17T11:42:02.006641+0000","last_active":"2024-09-17T11:42:26.814752+0000","last_peered":"2024-09-17T11:42:26.814752+0000","last_clean":"2024-09-17T11:42:26.814752+0000","last_became_active":"2024-09-17T11:42:02.006300+0000","last_became_peered":"2024-09-17T11:42:02.006300+0000","last_unstale":"2024-09-17T11:42:26.814752+0000","last_undegraded":"2024-09-17T11:42:26.814752+0000","last_fullsized":"2024-09-17T11:42:26.814752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:07.505904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"61'18","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428935+0000","last_change":"2024-09-17T11:41:58.206130+0000","last_active":"2024-09-17T11:42:26.428935+0000","last_peered":"2024-09-17T11:42:26.428935+0000","last_clean":"2024-09-17T11:42:26.428935+0000","last_became_active":"2024-09-17T11:41:58.205681+0000","last_became_peered":"2024-09-17T11:41:58.205681+0000","last_unstale":"2024-09-17T11:42:26.428935+0000","last_undegraded":"2024-09-17T11:42:26.428935+0000","last_fullsized":"2024-09-17T11:42:26.428935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:10.784028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429133+0000","last_change":"2024-09-17T11:41:55.984073+0000","last_active":"2024-09-17T11:42:26.429133+0000","last_peered":"2024-09-17T11:42:26.429133+0000","last_clean":"2024-09-17T11:42:26.429133+0000","last_became_active":"2024-09-17T11:41:55.983821+0000","last_became_peered":"2024-09-17T11:41:55.983821+0000","last_unstale":"2024-09-17T11:42:26.429133+0000","last_undegraded":"2024-09-17T11:42:26.429133+0000","last_fullsized":"2024-09-17T11:42:26.429133+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:28:14.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813742+0000","last_change":"2024-09-17T11:41:59.990838+0000","last_active":"2024-09-17T11:42:26.813742+0000","last_peered":"2024-09-17T11:42:26.813742+0000","last_clean":"2024-09-17T11:42:26.813742+0000","last_became_active":"2024-09-17T11:41:59.990549+0000","last_became_peered":"2024-09-17T11:41:59.990549+0000","last_unstale":"2024-09-17T11:42:26.813742+0000","last_undegraded":"2024-09-17T11:42:26.813742+0000","last_fullsized":"2024-09-17T11:42:26.813742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:42.605754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814343+0000","last_change":"2024-09-17T11:42:02.004328+0000","last_active":"2024-09-17T11:42:26.814343+0000","last_peered":"2024-09-17T11:42:26.814343+0000","last_clean":"2024-09-17T11:42:26.814343+0000","last_became_active":"2024-09-17T11:42:02.003378+0000","last_became_peered":"2024-09-17T11:42:02.003378+0000","last_unstale":"2024-09-17T11:42:26.814343+0000","last_undegraded":"2024-09-17T11:42:26.814343+0000","last_fullsized":"2024-09-17T11:42:26.814343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:47.129106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"61'14","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813834+0000","last_change":"2024-09-17T11:41:57.985302+0000","last_active":"2024-09-17T11:42:26.813834+0000","last_peered":"2024-09-17T11:42:26.813834+0000","last_clean":"2024-09-17T11:42:26.813834+0000","last_became_active":"2024-09-17T11:41:57.985005+0000","last_became_peered":"2024-09-17T11:41:57.985005+0000","last_unstale":"2024-09-17T11:42:26.813834+0000","last_undegraded":"2024-09-17T11:42:26.813834+0000","last_fullsized":"2024-09-17T11:42:26.813834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:13.712944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426721+0000","last_change":"2024-09-17T11:41:55.984977+0000","last_active":"2024-09-17T11:42:26.426721+0000","last_peered":"2024-09-17T11:42:26.426721+0000","last_clean":"2024-09-17T11:42:26.426721+0000","last_became_active":"2024-09-17T11:41:55.984696+0000","last_became_peered":"2024-09-17T11:41:55.984696+0000","last_unstale":"2024-09-17T11:42:26.426721+0000","last_undegraded":"2024-09-17T11:42:26.426721+0000","last_fullsized":"2024-09-17T11:42:26.426721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:06.534112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041185+0000","last_change":"2024-09-17T11:41:59.991503+0000","last_active":"2024-09-17T11:42:31.041185+0000","last_peered":"2024-09-17T11:42:31.041185+0000","last_clean":"2024-09-17T11:42:31.041185+0000","last_became_active":"2024-09-17T11:41:59.991159+0000","last_became_peered":"2024-09-17T11:41:59.991159+0000","last_unstale":"2024-09-17T11:42:31.041185+0000","last_undegraded":"2024-09-17T11:42:31.041185+0000","last_fullsized":"2024-09-17T11:42:31.041185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:56.635177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426410+0000","last_change":"2024-09-17T11:42:02.004031+0000","last_active":"2024-09-17T11:42:26.426410+0000","last_peered":"2024-09-17T11:42:26.426410+0000","last_clean":"2024-09-17T11:42:26.426410+0000","last_became_active":"2024-09-17T11:42:02.003732+0000","last_became_peered":"2024-09-17T11:42:02.003732+0000","last_unstale":"2024-09-17T11:42:26.426410+0000","last_undegraded":"2024-09-17T11:42:26.426410+0000","last_fullsized":"2024-09-17T11:42:26.426410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:29.575945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813961+0000","last_change":"2024-09-17T11:41:57.990483+0000","last_active":"2024-09-17T11:42:26.813961+0000","last_peered":"2024-09-17T11:42:26.813961+0000","last_clean":"2024-09-17T11:42:26.813961+0000","last_became_active":"2024-09-17T11:41:57.990131+0000","last_became_peered":"2024-09-17T11:41:57.990131+0000","last_unstale":"2024-09-17T11:42:26.813961+0000","last_undegraded":"2024-09-17T11:42:26.813961+0000","last_fullsized":"2024-09-17T11:42:26.813961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815297+0000","last_change":"2024-09-17T11:41:55.987143+0000","last_active":"2024-09-17T11:42:26.815297+0000","last_peered":"2024-09-17T11:42:26.815297+0000","last_clean":"2024-09-17T11:42:26.815297+0000","last_became_active":"2024-09-17T11:41:55.986842+0000","last_became_peered":"2024-09-17T11:41:55.986842+0000","last_unstale":"2024-09-17T11:42:26.815297+0000","last_undegraded":"2024-09-17T11:42:26.815297+0000","last_fullsized":"2024-09-17T11:42:26.815297+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:15.724779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041065+0000","last_change":"2024-09-17T11:41:59.996499+0000","last_active":"2024-09-17T11:42:31.041065+0000","last_peered":"2024-09-17T11:42:31.041065+0000","last_clean":"2024-09-17T11:42:31.041065+0000","last_became_active":"2024-09-17T11:41:59.996193+0000","last_became_peered":"2024-09-17T11:41:59.996193+0000","last_unstale":"2024-09-17T11:42:31.041065+0000","last_undegraded":"2024-09-17T11:42:31.041065+0000","last_fullsized":"2024-09-17T11:42:31.041065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:55.251678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424917+0000","last_change":"2024-09-17T11:42:01.996040+0000","last_active":"2024-09-17T11:42:26.424917+0000","last_peered":"2024-09-17T11:42:26.424917+0000","last_clean":"2024-09-17T11:42:26.424917+0000","last_became_active":"2024-09-17T11:42:01.995685+0000","last_became_peered":"2024-09-17T11:42:01.995685+0000","last_unstale":"2024-09-17T11:42:26.424917+0000","last_undegraded":"2024-09-17T11:42:26.424917+0000","last_fullsized":"2024-09-17T11:42:26.424917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:05.593119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"61'19","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425679+0000","last_change":"2024-09-17T11:41:58.208549+0000","last_active":"2024-09-17T11:42:26.425679+0000","last_peered":"2024-09-17T11:42:26.425679+0000","last_clean":"2024-09-17T11:42:26.425679+0000","last_became_active":"2024-09-17T11:41:58.208146+0000","last_became_peered":"2024-09-17T11:41:58.208146+0000","last_unstale":"2024-09-17T11:42:26.425679+0000","last_undegraded":"2024-09-17T11:42:26.425679+0000","last_fullsized":"2024-09-17T11:42:26.425679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:11:41.730484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814896+0000","last_change":"2024-09-17T11:41:55.986941+0000","last_active":"2024-09-17T11:42:26.814896+0000","last_peered":"2024-09-17T11:42:26.814896+0000","last_clean":"2024-09-17T11:42:26.814896+0000","last_became_active":"2024-09-17T11:41:55.986631+0000","last_became_peered":"2024-09-17T11:41:55.986631+0000","last_unstale":"2024-09-17T11:42:26.814896+0000","last_undegraded":"2024-09-17T11:42:26.814896+0000","last_fullsized":"2024-09-17T11:42:26.814896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:39.670685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815299+0000","last_change":"2024-09-17T11:41:59.994127+0000","last_active":"2024-09-17T11:42:26.815299+0000","last_peered":"2024-09-17T11:42:26.815299+0000","last_clean":"2024-09-17T11:42:26.815299+0000","last_became_active":"2024-09-17T11:41:59.993856+0000","last_became_peered":"2024-09-17T11:41:59.993856+0000","last_unstale":"2024-09-17T11:42:26.815299+0000","last_undegraded":"2024-09-17T11:42:26.815299+0000","last_fullsized":"2024-09-17T11:42:26.815299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:43.851796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813729+0000","last_change":"2024-09-17T11:42:01.990969+0000","last_active":"2024-09-17T11:42:26.813729+0000","last_peered":"2024-09-17T11:42:26.813729+0000","last_clean":"2024-09-17T11:42:26.813729+0000","last_became_active":"2024-09-17T11:42:01.990146+0000","last_became_peered":"2024-09-17T11:42:01.990146+0000","last_unstale":"2024-09-17T11:42:26.813729+0000","last_undegraded":"2024-09-17T11:42:26.813729+0000","last_fullsized":"2024-09-17T11:42:26.813729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:22.071049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"61'28","reported_seq":83,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041070+0000","last_change":"2024-09-17T11:41:57.990035+0000","last_active":"2024-09-17T11:42:31.041070+0000","last_peered":"2024-09-17T11:42:31.041070+0000","last_clean":"2024-09-17T11:42:31.041070+0000","last_became_active":"2024-09-17T11:41:57.989715+0000","last_became_peered":"2024-09-17T11:41:57.989715+0000","last_unstale":"2024-09-17T11:42:31.041070+0000","last_undegraded":"2024-09-17T11:42:31.041070+0000","last_fullsized":"2024-09-17T11:42:31.041070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:51.024351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814654+0000","last_change":"2024-09-17T11:41:55.985396+0000","last_active":"2024-09-17T11:42:26.814654+0000","last_peered":"2024-09-17T11:42:26.814654+0000","last_clean":"2024-09-17T11:42:26.814654+0000","last_became_active":"2024-09-17T11:41:55.984630+0000","last_became_peered":"2024-09-17T11:41:55.984630+0000","last_unstale":"2024-09-17T11:42:26.814654+0000","last_undegraded":"2024-09-17T11:42:26.814654+0000","last_fullsized":"2024-09-17T11:42:26.814654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:47:27.303630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815098+0000","last_change":"2024-09-17T11:41:57.983938+0000","last_active":"2024-09-17T11:42:26.815098+0000","last_peered":"2024-09-17T11:42:26.815098+0000","last_clean":"2024-09-17T11:42:26.815098+0000","last_became_active":"2024-09-17T11:41:54.975540+0000","last_became_peered":"2024-09-17T11:41:54.975540+0000","last_unstale":"2024-09-17T11:42:26.815098+0000","last_undegraded":"2024-09-17T11:42:26.815098+0000","last_fullsized":"2024-09-17T11:42:26.815098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:57.679692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022569360000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425792+0000","last_change":"2024-09-17T11:41:59.996303+0000","last_active":"2024-09-17T11:42:26.425792+0000","last_peered":"2024-09-17T11:42:26.425792+0000","last_clean":"2024-09-17T11:42:26.425792+0000","last_became_active":"2024-09-17T11:41:59.994936+0000","last_became_peered":"2024-09-17T11:41:59.994936+0000","last_unstale":"2024-09-17T11:42:26.425792+0000","last_undegraded":"2024-09-17T11:42:26.425792+0000","last_fullsized":"2024-09-17T11:42:26.425792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:18.022005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428293+0000","last_change":"2024-09-17T11:42:02.000936+0000","last_active":"2024-09-17T11:42:26.428293+0000","last_peered":"2024-09-17T11:42:26.428293+0000","last_clean":"2024-09-17T11:42:26.428293+0000","last_became_active":"2024-09-17T11:42:02.000592+0000","last_became_peered":"2024-09-17T11:42:02.000592+0000","last_unstale":"2024-09-17T11:42:26.428293+0000","last_undegraded":"2024-09-17T11:42:26.428293+0000","last_fullsized":"2024-09-17T11:42:26.428293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:39.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'13","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815136+0000","last_change":"2024-09-17T11:41:57.993050+0000","last_active":"2024-09-17T11:42:26.815136+0000","last_peered":"2024-09-17T11:42:26.815136+0000","last_clean":"2024-09-17T11:42:26.815136+0000","last_became_active":"2024-09-17T11:41:57.992660+0000","last_became_peered":"2024-09-17T11:41:57.992660+0000","last_unstale":"2024-09-17T11:42:26.815136+0000","last_undegraded":"2024-09-17T11:42:26.815136+0000","last_fullsized":"2024-09-17T11:42:26.815136+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:24:47.286666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813899+0000","last_change":"2024-09-17T11:41:55.987763+0000","last_active":"2024-09-17T11:42:26.813899+0000","last_peered":"2024-09-17T11:42:26.813899+0000","last_clean":"2024-09-17T11:42:26.813899+0000","last_became_active":"2024-09-17T11:41:55.987463+0000","last_became_peered":"2024-09-17T11:41:55.987463+0000","last_unstale":"2024-09-17T11:42:26.813899+0000","last_undegraded":"2024-09-17T11:42:26.813899+0000","last_fullsized":"2024-09-17T11:42:26.813899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:19.050443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815394+0000","last_change":"2024-09-17T11:41:57.977137+0000","last_active":"2024-09-17T11:42:26.815394+0000","last_peered":"2024-09-17T11:42:26.815394+0000","last_clean":"2024-09-17T11:42:26.815394+0000","last_became_active":"2024-09-17T11:41:54.977934+0000","last_became_peered":"2024-09-17T11:41:54.977934+0000","last_unstale":"2024-09-17T11:42:26.815394+0000","last_undegraded":"2024-09-17T11:42:26.815394+0000","last_fullsized":"2024-09-17T11:42:26.815394+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:42:38.476238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088804800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814665+0000","last_change":"2024-09-17T11:41:59.989964+0000","last_active":"2024-09-17T11:42:26.814665+0000","last_peered":"2024-09-17T11:42:26.814665+0000","last_clean":"2024-09-17T11:42:26.814665+0000","last_became_active":"2024-09-17T11:41:59.989650+0000","last_became_peered":"2024-09-17T11:41:59.989650+0000","last_unstale":"2024-09-17T11:42:26.814665+0000","last_undegraded":"2024-09-17T11:42:26.814665+0000","last_fullsized":"2024-09-17T11:42:26.814665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:35.426175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426534+0000","last_change":"2024-09-17T11:42:02.007937+0000","last_active":"2024-09-17T11:42:26.426534+0000","last_peered":"2024-09-17T11:42:26.426534+0000","last_clean":"2024-09-17T11:42:26.426534+0000","last_became_active":"2024-09-17T11:42:02.007385+0000","last_became_peered":"2024-09-17T11:42:02.007385+0000","last_unstale":"2024-09-17T11:42:26.426534+0000","last_undegraded":"2024-09-17T11:42:26.426534+0000","last_fullsized":"2024-09-17T11:42:26.426534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:01.632872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":44,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426580+0000","last_change":"2024-09-17T11:41:57.994215+0000","last_active":"2024-09-17T11:42:26.426580+0000","last_peered":"2024-09-17T11:42:26.426580+0000","last_clean":"2024-09-17T11:42:26.426580+0000","last_became_active":"2024-09-17T11:41:57.993899+0000","last_became_peered":"2024-09-17T11:41:57.993899+0000","last_unstale":"2024-09-17T11:42:26.426580+0000","last_undegraded":"2024-09-17T11:42:26.426580+0000","last_fullsized":"2024-09-17T11:42:26.426580+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:33:38.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425203+0000","last_change":"2024-09-17T11:41:55.985987+0000","last_active":"2024-09-17T11:42:26.425203+0000","last_peered":"2024-09-17T11:42:26.425203+0000","last_clean":"2024-09-17T11:42:26.425203+0000","last_became_active":"2024-09-17T11:41:55.985672+0000","last_became_peered":"2024-09-17T11:41:55.985672+0000","last_unstale":"2024-09-17T11:42:26.425203+0000","last_undegraded":"2024-09-17T11:42:26.425203+0000","last_fullsized":"2024-09-17T11:42:26.425203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:34:27.561847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:36.228216+0000","last_change":"2024-09-17T11:41:58.206699+0000","last_active":"2024-09-17T11:42:36.228216+0000","last_peered":"2024-09-17T11:42:36.228216+0000","last_clean":"2024-09-17T11:42:36.228216+0000","last_became_active":"2024-09-17T11:41:54.979092+0000","last_became_peered":"2024-09-17T11:41:54.979092+0000","last_unstale":"2024-09-17T11:42:36.228216+0000","last_undegraded":"2024-09-17T11:42:36.228216+0000","last_fullsized":"2024-09-17T11:42:36.228216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:33:49.383136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035626,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815545+0000","last_change":"2024-09-17T11:41:59.996698+0000","last_active":"2024-09-17T11:42:26.815545+0000","last_peered":"2024-09-17T11:42:26.815545+0000","last_clean":"2024-09-17T11:42:26.815545+0000","last_became_active":"2024-09-17T11:41:59.996421+0000","last_became_peered":"2024-09-17T11:41:59.996421+0000","last_unstale":"2024-09-17T11:42:26.815545+0000","last_undegraded":"2024-09-17T11:42:26.815545+0000","last_fullsized":"2024-09-17T11:42:26.815545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:44.901238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814440+0000","last_change":"2024-09-17T11:42:02.002018+0000","last_active":"2024-09-17T11:42:26.814440+0000","last_peered":"2024-09-17T11:42:26.814440+0000","last_clean":"2024-09-17T11:42:26.814440+0000","last_became_active":"2024-09-17T11:42:02.001686+0000","last_became_peered":"2024-09-17T11:42:02.001686+0000","last_unstale":"2024-09-17T11:42:26.814440+0000","last_undegraded":"2024-09-17T11:42:26.814440+0000","last_fullsized":"2024-09-17T11:42:26.814440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:15.445854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'16","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041191+0000","last_change":"2024-09-17T11:41:58.207627+0000","last_active":"2024-09-17T11:42:31.041191+0000","last_peered":"2024-09-17T11:42:31.041191+0000","last_clean":"2024-09-17T11:42:31.041191+0000","last_became_active":"2024-09-17T11:41:58.207249+0000","last_became_peered":"2024-09-17T11:41:58.207249+0000","last_unstale":"2024-09-17T11:42:31.041191+0000","last_undegraded":"2024-09-17T11:42:31.041191+0000","last_fullsized":"2024-09-17T11:42:31.041191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:42.670782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429440+0000","last_change":"2024-09-17T11:41:55.983112+0000","last_active":"2024-09-17T11:42:26.429440+0000","last_peered":"2024-09-17T11:42:26.429440+0000","last_clean":"2024-09-17T11:42:26.429440+0000","last_became_active":"2024-09-17T11:41:55.982889+0000","last_became_peered":"2024-09-17T11:41:55.982889+0000","last_unstale":"2024-09-17T11:42:26.429440+0000","last_undegraded":"2024-09-17T11:42:26.429440+0000","last_fullsized":"2024-09-17T11:42:26.429440+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:51:45.874787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"63'79","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:28.887229+0000","last_change":"2024-09-17T11:41:48.977527+0000","last_active":"2024-09-17T11:42:28.887229+0000","last_peered":"2024-09-17T11:42:28.887229+0000","last_clean":"2024-09-17T11:42:28.887229+0000","last_became_active":"2024-09-17T11:41:48.971851+0000","last_became_peered":"2024-09-17T11:41:48.971851+0000","last_unstale":"2024-09-17T11:42:28.887229+0000","last_undegraded":"2024-09-17T11:42:28.887229+0000","last_fullsized":"2024-09-17T11:42:28.887229+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:39:43.074460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:39:43.074460+0000","last_clean_scrub_stamp":"2024-09-17T11:39:43.074460+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:22.643693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426946+0000","last_change":"2024-09-17T11:41:59.990248+0000","last_active":"2024-09-17T11:42:26.426946+0000","last_peered":"2024-09-17T11:42:26.426946+0000","last_clean":"2024-09-17T11:42:26.426946+0000","last_became_active":"2024-09-17T11:41:59.989889+0000","last_became_peered":"2024-09-17T11:41:59.989889+0000","last_unstale":"2024-09-17T11:42:26.426946+0000","last_undegraded":"2024-09-17T11:42:26.426946+0000","last_fullsized":"2024-09-17T11:42:26.426946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:16.851610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815371+0000","last_change":"2024-09-17T11:42:02.002139+0000","last_active":"2024-09-17T11:42:26.815371+0000","last_peered":"2024-09-17T11:42:26.815371+0000","last_clean":"2024-09-17T11:42:26.815371+0000","last_became_active":"2024-09-17T11:42:02.001853+0000","last_became_peered":"2024-09-17T11:42:02.001853+0000","last_unstale":"2024-09-17T11:42:26.815371+0000","last_undegraded":"2024-09-17T11:42:26.815371+0000","last_fullsized":"2024-09-17T11:42:26.815371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:41:57.655567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814830+0000","last_change":"2024-09-17T11:41:57.989950+0000","last_active":"2024-09-17T11:42:26.814830+0000","last_peered":"2024-09-17T11:42:26.814830+0000","last_clean":"2024-09-17T11:42:26.814830+0000","last_became_active":"2024-09-17T11:41:57.989642+0000","last_became_peered":"2024-09-17T11:41:57.989642+0000","last_unstale":"2024-09-17T11:42:26.814830+0000","last_undegraded":"2024-09-17T11:42:26.814830+0000","last_fullsized":"2024-09-17T11:42:26.814830+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:01.799841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814295+0000","last_change":"2024-09-17T11:41:55.989222+0000","last_active":"2024-09-17T11:42:26.814295+0000","last_peered":"2024-09-17T11:42:26.814295+0000","last_clean":"2024-09-17T11:42:26.814295+0000","last_became_active":"2024-09-17T11:41:55.988674+0000","last_became_peered":"2024-09-17T11:41:55.988674+0000","last_unstale":"2024-09-17T11:42:26.814295+0000","last_undegraded":"2024-09-17T11:42:26.814295+0000","last_fullsized":"2024-09-17T11:42:26.814295+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:08.566186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815237+0000","last_change":"2024-09-17T11:41:59.988719+0000","last_active":"2024-09-17T11:42:26.815237+0000","last_peered":"2024-09-17T11:42:26.815237+0000","last_clean":"2024-09-17T11:42:26.815237+0000","last_became_active":"2024-09-17T11:41:59.988463+0000","last_became_peered":"2024-09-17T11:41:59.988463+0000","last_unstale":"2024-09-17T11:42:26.815237+0000","last_undegraded":"2024-09-17T11:42:26.815237+0000","last_fullsized":"2024-09-17T11:42:26.815237+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:50.655214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427799+0000","last_change":"2024-09-17T11:42:01.997964+0000","last_active":"2024-09-17T11:42:26.427799+0000","last_peered":"2024-09-17T11:42:26.427799+0000","last_clean":"2024-09-17T11:42:26.427799+0000","last_became_active":"2024-09-17T11:42:01.997672+0000","last_became_peered":"2024-09-17T11:42:01.997672+0000","last_unstale":"2024-09-17T11:42:26.427799+0000","last_undegraded":"2024-09-17T11:42:26.427799+0000","last_fullsized":"2024-09-17T11:42:26.427799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:20:42.353045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814498+0000","last_change":"2024-09-17T11:41:57.991635+0000","last_active":"2024-09-17T11:42:26.814498+0000","last_peered":"2024-09-17T11:42:26.814498+0000","last_clean":"2024-09-17T11:42:26.814498+0000","last_became_active":"2024-09-17T11:41:57.990969+0000","last_became_peered":"2024-09-17T11:41:57.990969+0000","last_unstale":"2024-09-17T11:42:26.814498+0000","last_undegraded":"2024-09-17T11:42:26.814498+0000","last_fullsized":"2024-09-17T11:42:26.814498+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:36.826155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815528+0000","last_change":"2024-09-17T11:41:55.979515+0000","last_active":"2024-09-17T11:42:26.815528+0000","last_peered":"2024-09-17T11:42:26.815528+0000","last_clean":"2024-09-17T11:42:26.815528+0000","last_became_active":"2024-09-17T11:41:55.979188+0000","last_became_peered":"2024-09-17T11:41:55.979188+0000","last_unstale":"2024-09-17T11:42:26.815528+0000","last_undegraded":"2024-09-17T11:42:26.815528+0000","last_fullsized":"2024-09-17T11:42:26.815528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:41:15.514646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814543+0000","last_change":"2024-09-17T11:41:59.998804+0000","last_active":"2024-09-17T11:42:26.814543+0000","last_peered":"2024-09-17T11:42:26.814543+0000","last_clean":"2024-09-17T11:42:26.814543+0000","last_became_active":"2024-09-17T11:41:59.998488+0000","last_became_peered":"2024-09-17T11:41:59.998488+0000","last_unstale":"2024-09-17T11:42:26.814543+0000","last_undegraded":"2024-09-17T11:42:26.814543+0000","last_fullsized":"2024-09-17T11:42:26.814543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:22:37.439485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814561+0000","last_change":"2024-09-17T11:42:01.998759+0000","last_active":"2024-09-17T11:42:26.814561+0000","last_peered":"2024-09-17T11:42:26.814561+0000","last_clean":"2024-09-17T11:42:26.814561+0000","last_became_active":"2024-09-17T11:42:01.998446+0000","last_became_peered":"2024-09-17T11:42:01.998446+0000","last_unstale":"2024-09-17T11:42:26.814561+0000","last_undegraded":"2024-09-17T11:42:26.814561+0000","last_fullsized":"2024-09-17T11:42:26.814561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:41.175648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814306+0000","last_change":"2024-09-17T11:41:57.987267+0000","last_active":"2024-09-17T11:42:26.814306+0000","last_peered":"2024-09-17T11:42:26.814306+0000","last_clean":"2024-09-17T11:42:26.814306+0000","last_became_active":"2024-09-17T11:41:57.986949+0000","last_became_peered":"2024-09-17T11:41:57.986949+0000","last_unstale":"2024-09-17T11:42:26.814306+0000","last_undegraded":"2024-09-17T11:42:26.814306+0000","last_fullsized":"2024-09-17T11:42:26.814306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:03.940212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429643+0000","last_change":"2024-09-17T11:41:55.983110+0000","last_active":"2024-09-17T11:42:26.429643+0000","last_peered":"2024-09-17T11:42:26.429643+0000","last_clean":"2024-09-17T11:42:26.429643+0000","last_became_active":"2024-09-17T11:41:55.982829+0000","last_became_peered":"2024-09-17T11:41:55.982829+0000","last_unstale":"2024-09-17T11:42:26.429643+0000","last_undegraded":"2024-09-17T11:42:26.429643+0000","last_fullsized":"2024-09-17T11:42:26.429643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:51.255636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041051+0000","last_change":"2024-09-17T11:41:59.990588+0000","last_active":"2024-09-17T11:42:31.041051+0000","last_peered":"2024-09-17T11:42:31.041051+0000","last_clean":"2024-09-17T11:42:31.041051+0000","last_became_active":"2024-09-17T11:41:59.990273+0000","last_became_peered":"2024-09-17T11:41:59.990273+0000","last_unstale":"2024-09-17T11:42:31.041051+0000","last_undegraded":"2024-09-17T11:42:31.041051+0000","last_fullsized":"2024-09-17T11:42:31.041051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:29:02.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813984+0000","last_change":"2024-09-17T11:42:02.003543+0000","last_active":"2024-09-17T11:42:26.813984+0000","last_peered":"2024-09-17T11:42:26.813984+0000","last_clean":"2024-09-17T11:42:26.813984+0000","last_became_active":"2024-09-17T11:42:02.003264+0000","last_became_peered":"2024-09-17T11:42:02.003264+0000","last_unstale":"2024-09-17T11:42:26.813984+0000","last_undegraded":"2024-09-17T11:42:26.813984+0000","last_fullsized":"2024-09-17T11:42:26.813984+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:14:04.659732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425578+0000","last_change":"2024-09-17T11:41:57.990006+0000","last_active":"2024-09-17T11:42:26.425578+0000","last_peered":"2024-09-17T11:42:26.425578+0000","last_clean":"2024-09-17T11:42:26.425578+0000","last_became_active":"2024-09-17T11:41:57.989716+0000","last_became_peered":"2024-09-17T11:41:57.989716+0000","last_unstale":"2024-09-17T11:42:26.425578+0000","last_undegraded":"2024-09-17T11:42:26.425578+0000","last_fullsized":"2024-09-17T11:42:26.425578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:07.100432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814796+0000","last_change":"2024-09-17T11:41:55.979389+0000","last_active":"2024-09-17T11:42:26.814796+0000","last_peered":"2024-09-17T11:42:26.814796+0000","last_clean":"2024-09-17T11:42:26.814796+0000","last_became_active":"2024-09-17T11:41:55.979072+0000","last_became_peered":"2024-09-17T11:41:55.979072+0000","last_unstale":"2024-09-17T11:42:26.814796+0000","last_undegraded":"2024-09-17T11:42:26.814796+0000","last_fullsized":"2024-09-17T11:42:26.814796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:35:27.267245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815366+0000","last_change":"2024-09-17T11:41:59.985802+0000","last_active":"2024-09-17T11:42:26.815366+0000","last_peered":"2024-09-17T11:42:26.815366+0000","last_clean":"2024-09-17T11:42:26.815366+0000","last_became_active":"2024-09-17T11:41:59.985285+0000","last_became_peered":"2024-09-17T11:41:59.985285+0000","last_unstale":"2024-09-17T11:42:26.815366+0000","last_undegraded":"2024-09-17T11:42:26.815366+0000","last_fullsized":"2024-09-17T11:42:26.815366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:38.527214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425106+0000","last_change":"2024-09-17T11:42:01.998275+0000","last_active":"2024-09-17T11:42:26.425106+0000","last_peered":"2024-09-17T11:42:26.425106+0000","last_clean":"2024-09-17T11:42:26.425106+0000","last_became_active":"2024-09-17T11:42:01.997994+0000","last_became_peered":"2024-09-17T11:42:01.997994+0000","last_unstale":"2024-09-17T11:42:26.425106+0000","last_undegraded":"2024-09-17T11:42:26.425106+0000","last_fullsized":"2024-09-17T11:42:26.425106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:54:09.619950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'19","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815365+0000","last_change":"2024-09-17T11:41:58.208112+0000","last_active":"2024-09-17T11:42:26.815365+0000","last_peered":"2024-09-17T11:42:26.815365+0000","last_clean":"2024-09-17T11:42:26.815365+0000","last_became_active":"2024-09-17T11:41:58.207290+0000","last_became_peered":"2024-09-17T11:41:58.207290+0000","last_unstale":"2024-09-17T11:42:26.815365+0000","last_undegraded":"2024-09-17T11:42:26.815365+0000","last_fullsized":"2024-09-17T11:42:26.815365+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:26.241882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426940+0000","last_change":"2024-09-17T11:41:55.985115+0000","last_active":"2024-09-17T11:42:26.426940+0000","last_peered":"2024-09-17T11:42:26.426940+0000","last_clean":"2024-09-17T11:42:26.426940+0000","last_became_active":"2024-09-17T11:41:55.984842+0000","last_became_peered":"2024-09-17T11:41:55.984842+0000","last_unstale":"2024-09-17T11:42:26.426940+0000","last_undegraded":"2024-09-17T11:42:26.426940+0000","last_fullsized":"2024-09-17T11:42:26.426940+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:50.329355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814042+0000","last_change":"2024-09-17T11:41:59.989276+0000","last_active":"2024-09-17T11:42:26.814042+0000","last_peered":"2024-09-17T11:42:26.814042+0000","last_clean":"2024-09-17T11:42:26.814042+0000","last_became_active":"2024-09-17T11:41:59.988964+0000","last_became_peered":"2024-09-17T11:41:59.988964+0000","last_unstale":"2024-09-17T11:42:26.814042+0000","last_undegraded":"2024-09-17T11:42:26.814042+0000","last_fullsized":"2024-09-17T11:42:26.814042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:36.955466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426216+0000","last_change":"2024-09-17T11:42:02.005488+0000","last_active":"2024-09-17T11:42:26.426216+0000","last_peered":"2024-09-17T11:42:26.426216+0000","last_clean":"2024-09-17T11:42:26.426216+0000","last_became_active":"2024-09-17T11:42:02.005175+0000","last_became_peered":"2024-09-17T11:42:02.005175+0000","last_unstale":"2024-09-17T11:42:26.426216+0000","last_undegraded":"2024-09-17T11:42:26.426216+0000","last_fullsized":"2024-09-17T11:42:26.426216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:32.740868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814159+0000","last_change":"2024-09-17T11:41:57.986300+0000","last_active":"2024-09-17T11:42:26.814159+0000","last_peered":"2024-09-17T11:42:26.814159+0000","last_clean":"2024-09-17T11:42:26.814159+0000","last_became_active":"2024-09-17T11:41:57.985853+0000","last_became_peered":"2024-09-17T11:41:57.985853+0000","last_unstale":"2024-09-17T11:42:26.814159+0000","last_undegraded":"2024-09-17T11:42:26.814159+0000","last_fullsized":"2024-09-17T11:42:26.814159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:54:54.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426980+0000","last_change":"2024-09-17T11:41:55.990216+0000","last_active":"2024-09-17T11:42:26.426980+0000","last_peered":"2024-09-17T11:42:26.426980+0000","last_clean":"2024-09-17T11:42:26.426980+0000","last_became_active":"2024-09-17T11:41:55.989967+0000","last_became_peered":"2024-09-17T11:41:55.989967+0000","last_unstale":"2024-09-17T11:42:26.426980+0000","last_undegraded":"2024-09-17T11:42:26.426980+0000","last_fullsized":"2024-09-17T11:42:26.426980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:34.401540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814409+0000","last_change":"2024-09-17T11:41:59.995275+0000","last_active":"2024-09-17T11:42:26.814409+0000","last_peered":"2024-09-17T11:42:26.814409+0000","last_clean":"2024-09-17T11:42:26.814409+0000","last_became_active":"2024-09-17T11:41:59.994712+0000","last_became_peered":"2024-09-17T11:41:59.994712+0000","last_unstale":"2024-09-17T11:42:26.814409+0000","last_undegraded":"2024-09-17T11:42:26.814409+0000","last_fullsized":"2024-09-17T11:42:26.814409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:21.580649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429270+0000","last_change":"2024-09-17T11:42:02.001168+0000","last_active":"2024-09-17T11:42:26.429270+0000","last_peered":"2024-09-17T11:42:26.429270+0000","last_clean":"2024-09-17T11:42:26.429270+0000","last_became_active":"2024-09-17T11:42:02.000257+0000","last_became_peered":"2024-09-17T11:42:02.000257+0000","last_unstale":"2024-09-17T11:42:26.429270+0000","last_undegraded":"2024-09-17T11:42:26.429270+0000","last_fullsized":"2024-09-17T11:42:26.429270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:43.861697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813617+0000","last_change":"2024-09-17T11:41:58.206453+0000","last_active":"2024-09-17T11:42:26.813617+0000","last_peered":"2024-09-17T11:42:26.813617+0000","last_clean":"2024-09-17T11:42:26.813617+0000","last_became_active":"2024-09-17T11:41:58.206166+0000","last_became_peered":"2024-09-17T11:41:58.206166+0000","last_unstale":"2024-09-17T11:42:26.813617+0000","last_undegraded":"2024-09-17T11:42:26.813617+0000","last_fullsized":"2024-09-17T11:42:26.813617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:47.145321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426828+0000","last_change":"2024-09-17T11:41:55.988303+0000","last_active":"2024-09-17T11:42:26.426828+0000","last_peered":"2024-09-17T11:42:26.426828+0000","last_clean":"2024-09-17T11:42:26.426828+0000","last_became_active":"2024-09-17T11:41:55.987994+0000","last_became_peered":"2024-09-17T11:41:55.987994+0000","last_unstale":"2024-09-17T11:42:26.426828+0000","last_undegraded":"2024-09-17T11:42:26.426828+0000","last_fullsized":"2024-09-17T11:42:26.426828+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:37.381057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041129+0000","last_change":"2024-09-17T11:41:59.992449+0000","last_active":"2024-09-17T11:42:31.041129+0000","last_peered":"2024-09-17T11:42:31.041129+0000","last_clean":"2024-09-17T11:42:31.041129+0000","last_became_active":"2024-09-17T11:41:59.992128+0000","last_became_peered":"2024-09-17T11:41:59.992128+0000","last_unstale":"2024-09-17T11:42:31.041129+0000","last_undegraded":"2024-09-17T11:42:31.041129+0000","last_fullsized":"2024-09-17T11:42:31.041129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:06.370600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813917+0000","last_change":"2024-09-17T11:42:01.997709+0000","last_active":"2024-09-17T11:42:26.813917+0000","last_peered":"2024-09-17T11:42:26.813917+0000","last_clean":"2024-09-17T11:42:26.813917+0000","last_became_active":"2024-09-17T11:42:01.997004+0000","last_became_peered":"2024-09-17T11:42:01.997004+0000","last_unstale":"2024-09-17T11:42:26.813917+0000","last_undegraded":"2024-09-17T11:42:26.813917+0000","last_fullsized":"2024-09-17T11:42:26.813917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:22.940365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813762+0000","last_change":"2024-09-17T11:41:55.980616+0000","last_active":"2024-09-17T11:42:26.813762+0000","last_peered":"2024-09-17T11:42:26.813762+0000","last_clean":"2024-09-17T11:42:26.813762+0000","last_became_active":"2024-09-17T11:41:55.980349+0000","last_became_peered":"2024-09-17T11:41:55.980349+0000","last_unstale":"2024-09-17T11:42:26.813762+0000","last_undegraded":"2024-09-17T11:42:26.813762+0000","last_fullsized":"2024-09-17T11:42:26.813762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:08:04.927608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'6","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428994+0000","last_change":"2024-09-17T11:41:58.207530+0000","last_active":"2024-09-17T11:42:26.428994+0000","last_peered":"2024-09-17T11:42:26.428994+0000","last_clean":"2024-09-17T11:42:26.428994+0000","last_became_active":"2024-09-17T11:41:58.207152+0000","last_became_peered":"2024-09-17T11:41:58.207152+0000","last_unstale":"2024-09-17T11:42:26.428994+0000","last_undegraded":"2024-09-17T11:42:26.428994+0000","last_fullsized":"2024-09-17T11:42:26.428994+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:08.737262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814598+0000","last_change":"2024-09-17T11:41:59.991365+0000","last_active":"2024-09-17T11:42:26.814598+0000","last_peered":"2024-09-17T11:42:26.814598+0000","last_clean":"2024-09-17T11:42:26.814598+0000","last_became_active":"2024-09-17T11:41:59.991058+0000","last_became_peered":"2024-09-17T11:41:59.991058+0000","last_unstale":"2024-09-17T11:42:26.814598+0000","last_undegraded":"2024-09-17T11:42:26.814598+0000","last_fullsized":"2024-09-17T11:42:26.814598+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:21:02.626705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426454+0000","last_change":"2024-09-17T11:42:02.001290+0000","last_active":"2024-09-17T11:42:26.426454+0000","last_peered":"2024-09-17T11:42:26.426454+0000","last_clean":"2024-09-17T11:42:26.426454+0000","last_became_active":"2024-09-17T11:42:02.000165+0000","last_became_peered":"2024-09-17T11:42:02.000165+0000","last_unstale":"2024-09-17T11:42:26.426454+0000","last_undegraded":"2024-09-17T11:42:26.426454+0000","last_fullsized":"2024-09-17T11:42:26.426454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:53.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426699+0000","last_change":"2024-09-17T11:41:58.209918+0000","last_active":"2024-09-17T11:42:26.426699+0000","last_peered":"2024-09-17T11:42:26.426699+0000","last_clean":"2024-09-17T11:42:26.426699+0000","last_became_active":"2024-09-17T11:41:58.209531+0000","last_became_peered":"2024-09-17T11:41:58.209531+0000","last_unstale":"2024-09-17T11:42:26.426699+0000","last_undegraded":"2024-09-17T11:42:26.426699+0000","last_fullsized":"2024-09-17T11:42:26.426699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:27.508359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426646+0000","last_change":"2024-09-17T11:41:55.989087+0000","last_active":"2024-09-17T11:42:26.426646+0000","last_peered":"2024-09-17T11:42:26.426646+0000","last_clean":"2024-09-17T11:42:26.426646+0000","last_became_active":"2024-09-17T11:41:55.988807+0000","last_became_peered":"2024-09-17T11:41:55.988807+0000","last_unstale":"2024-09-17T11:42:26.426646+0000","last_undegraded":"2024-09-17T11:42:26.426646+0000","last_fullsized":"2024-09-17T11:42:26.426646+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:42.340712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428870+0000","last_change":"2024-09-17T11:41:59.995712+0000","last_active":"2024-09-17T11:42:26.428870+0000","last_peered":"2024-09-17T11:42:26.428870+0000","last_clean":"2024-09-17T11:42:26.428870+0000","last_became_active":"2024-09-17T11:41:59.995390+0000","last_became_peered":"2024-09-17T11:41:59.995390+0000","last_unstale":"2024-09-17T11:42:26.428870+0000","last_undegraded":"2024-09-17T11:42:26.428870+0000","last_fullsized":"2024-09-17T11:42:26.428870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:30:45.824979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814112+0000","last_change":"2024-09-17T11:42:01.999831+0000","last_active":"2024-09-17T11:42:26.814112+0000","last_peered":"2024-09-17T11:42:26.814112+0000","last_clean":"2024-09-17T11:42:26.814112+0000","last_became_active":"2024-09-17T11:42:01.998381+0000","last_became_peered":"2024-09-17T11:42:01.998381+0000","last_unstale":"2024-09-17T11:42:26.814112+0000","last_undegraded":"2024-09-17T11:42:26.814112+0000","last_fullsized":"2024-09-17T11:42:26.814112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:07.503273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815097+0000","last_change":"2024-09-17T11:41:58.208071+0000","last_active":"2024-09-17T11:42:26.815097+0000","last_peered":"2024-09-17T11:42:26.815097+0000","last_clean":"2024-09-17T11:42:26.815097+0000","last_became_active":"2024-09-17T11:41:58.207692+0000","last_became_peered":"2024-09-17T11:41:58.207692+0000","last_unstale":"2024-09-17T11:42:26.815097+0000","last_undegraded":"2024-09-17T11:42:26.815097+0000","last_fullsized":"2024-09-17T11:42:26.815097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:54.089576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425245+0000","last_change":"2024-09-17T11:41:55.984519+0000","last_active":"2024-09-17T11:42:26.425245+0000","last_peered":"2024-09-17T11:42:26.425245+0000","last_clean":"2024-09-17T11:42:26.425245+0000","last_became_active":"2024-09-17T11:41:55.984191+0000","last_became_peered":"2024-09-17T11:41:55.984191+0000","last_unstale":"2024-09-17T11:42:26.425245+0000","last_undegraded":"2024-09-17T11:42:26.425245+0000","last_fullsized":"2024-09-17T11:42:26.425245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:20.663913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041493+0000","last_change":"2024-09-17T11:41:59.990171+0000","last_active":"2024-09-17T11:42:31.041493+0000","last_peered":"2024-09-17T11:42:31.041493+0000","last_clean":"2024-09-17T11:42:31.041493+0000","last_became_active":"2024-09-17T11:41:59.989863+0000","last_became_peered":"2024-09-17T11:41:59.989863+0000","last_unstale":"2024-09-17T11:42:31.041493+0000","last_undegraded":"2024-09-17T11:42:31.041493+0000","last_fullsized":"2024-09-17T11:42:31.041493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:53:50.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814851+0000","last_change":"2024-09-17T11:42:02.006964+0000","last_active":"2024-09-17T11:42:26.814851+0000","last_peered":"2024-09-17T11:42:26.814851+0000","last_clean":"2024-09-17T11:42:26.814851+0000","last_became_active":"2024-09-17T11:42:02.006674+0000","last_became_peered":"2024-09-17T11:42:02.006674+0000","last_unstale":"2024-09-17T11:42:26.814851+0000","last_undegraded":"2024-09-17T11:42:26.814851+0000","last_fullsized":"2024-09-17T11:42:26.814851+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:40.991561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429948+0000","last_change":"2024-09-17T11:41:58.206871+0000","last_active":"2024-09-17T11:42:26.429948+0000","last_peered":"2024-09-17T11:42:26.429948+0000","last_clean":"2024-09-17T11:42:26.429948+0000","last_became_active":"2024-09-17T11:41:58.206136+0000","last_became_peered":"2024-09-17T11:41:58.206136+0000","last_unstale":"2024-09-17T11:42:26.429948+0000","last_undegraded":"2024-09-17T11:42:26.429948+0000","last_fullsized":"2024-09-17T11:42:26.429948+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:00.937087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426746+0000","last_change":"2024-09-17T11:41:55.988710+0000","last_active":"2024-09-17T11:42:26.426746+0000","last_peered":"2024-09-17T11:42:26.426746+0000","last_clean":"2024-09-17T11:42:26.426746+0000","last_became_active":"2024-09-17T11:41:55.988318+0000","last_became_peered":"2024-09-17T11:41:55.988318+0000","last_unstale":"2024-09-17T11:42:26.426746+0000","last_undegraded":"2024-09-17T11:42:26.426746+0000","last_fullsized":"2024-09-17T11:42:26.426746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:16.376705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041200+0000","last_change":"2024-09-17T11:41:59.995602+0000","last_active":"2024-09-17T11:42:31.041200+0000","last_peered":"2024-09-17T11:42:31.041200+0000","last_clean":"2024-09-17T11:42:31.041200+0000","last_became_active":"2024-09-17T11:41:59.995180+0000","last_became_peered":"2024-09-17T11:41:59.995180+0000","last_unstale":"2024-09-17T11:42:31.041200+0000","last_undegraded":"2024-09-17T11:42:31.041200+0000","last_fullsized":"2024-09-17T11:42:31.041200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426347+0000","last_change":"2024-09-17T11:42:02.001484+0000","last_active":"2024-09-17T11:42:26.426347+0000","last_peered":"2024-09-17T11:42:26.426347+0000","last_clean":"2024-09-17T11:42:26.426347+0000","last_became_active":"2024-09-17T11:42:02.001124+0000","last_became_peered":"2024-09-17T11:42:02.001124+0000","last_unstale":"2024-09-17T11:42:26.426347+0000","last_undegraded":"2024-09-17T11:42:26.426347+0000","last_fullsized":"2024-09-17T11:42:26.426347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:12.246081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814151+0000","last_change":"2024-09-17T11:41:57.990684+0000","last_active":"2024-09-17T11:42:26.814151+0000","last_peered":"2024-09-17T11:42:26.814151+0000","last_clean":"2024-09-17T11:42:26.814151+0000","last_became_active":"2024-09-17T11:41:57.990347+0000","last_became_peered":"2024-09-17T11:41:57.990347+0000","last_unstale":"2024-09-17T11:42:26.814151+0000","last_undegraded":"2024-09-17T11:42:26.814151+0000","last_fullsized":"2024-09-17T11:42:26.814151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:22.170034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814459+0000","last_change":"2024-09-17T11:41:55.986557+0000","last_active":"2024-09-17T11:42:26.814459+0000","last_peered":"2024-09-17T11:42:26.814459+0000","last_clean":"2024-09-17T11:42:26.814459+0000","last_became_active":"2024-09-17T11:41:55.986282+0000","last_became_peered":"2024-09-17T11:41:55.986282+0000","last_unstale":"2024-09-17T11:42:26.814459+0000","last_undegraded":"2024-09-17T11:42:26.814459+0000","last_fullsized":"2024-09-17T11:42:26.814459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:07.660835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813800+0000","last_change":"2024-09-17T11:41:59.995587+0000","last_active":"2024-09-17T11:42:26.813800+0000","last_peered":"2024-09-17T11:42:26.813800+0000","last_clean":"2024-09-17T11:42:26.813800+0000","last_became_active":"2024-09-17T11:41:59.995304+0000","last_became_peered":"2024-09-17T11:41:59.995304+0000","last_unstale":"2024-09-17T11:42:26.813800+0000","last_undegraded":"2024-09-17T11:42:26.813800+0000","last_fullsized":"2024-09-17T11:42:26.813800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:55.392903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427445+0000","last_change":"2024-09-17T11:42:01.998771+0000","last_active":"2024-09-17T11:42:26.427445+0000","last_peered":"2024-09-17T11:42:26.427445+0000","last_clean":"2024-09-17T11:42:26.427445+0000","last_became_active":"2024-09-17T11:42:01.998466+0000","last_became_peered":"2024-09-17T11:42:01.998466+0000","last_unstale":"2024-09-17T11:42:26.427445+0000","last_undegraded":"2024-09-17T11:42:26.427445+0000","last_fullsized":"2024-09-17T11:42:26.427445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:17:23.282240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814159+0000","last_change":"2024-09-17T11:41:57.996397+0000","last_active":"2024-09-17T11:42:26.814159+0000","last_peered":"2024-09-17T11:42:26.814159+0000","last_clean":"2024-09-17T11:42:26.814159+0000","last_became_active":"2024-09-17T11:41:57.996119+0000","last_became_peered":"2024-09-17T11:41:57.996119+0000","last_unstale":"2024-09-17T11:42:26.814159+0000","last_undegraded":"2024-09-17T11:42:26.814159+0000","last_fullsized":"2024-09-17T11:42:26.814159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:32.721330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426769+0000","last_change":"2024-09-17T11:41:55.988135+0000","last_active":"2024-09-17T11:42:26.426769+0000","last_peered":"2024-09-17T11:42:26.426769+0000","last_clean":"2024-09-17T11:42:26.426769+0000","last_became_active":"2024-09-17T11:41:55.987846+0000","last_became_peered":"2024-09-17T11:41:55.987846+0000","last_unstale":"2024-09-17T11:42:26.426769+0000","last_undegraded":"2024-09-17T11:42:26.426769+0000","last_fullsized":"2024-09-17T11:42:26.426769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:03.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429376+0000","last_change":"2024-09-17T11:41:59.993716+0000","last_active":"2024-09-17T11:42:26.429376+0000","last_peered":"2024-09-17T11:42:26.429376+0000","last_clean":"2024-09-17T11:42:26.429376+0000","last_became_active":"2024-09-17T11:41:59.993373+0000","last_became_peered":"2024-09-17T11:41:59.993373+0000","last_unstale":"2024-09-17T11:42:26.429376+0000","last_undegraded":"2024-09-17T11:42:26.429376+0000","last_fullsized":"2024-09-17T11:42:26.429376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:46:29.135614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424604+0000","last_change":"2024-09-17T11:42:01.999007+0000","last_active":"2024-09-17T11:42:26.424604+0000","last_peered":"2024-09-17T11:42:26.424604+0000","last_clean":"2024-09-17T11:42:26.424604+0000","last_became_active":"2024-09-17T11:42:01.998694+0000","last_became_peered":"2024-09-17T11:42:01.998694+0000","last_unstale":"2024-09-17T11:42:26.424604+0000","last_undegraded":"2024-09-17T11:42:26.424604+0000","last_fullsized":"2024-09-17T11:42:26.424604+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:30:09.400297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427627+0000","last_change":"2024-09-17T11:41:58.206059+0000","last_active":"2024-09-17T11:42:26.427627+0000","last_peered":"2024-09-17T11:42:26.427627+0000","last_clean":"2024-09-17T11:42:26.427627+0000","last_became_active":"2024-09-17T11:41:58.205660+0000","last_became_peered":"2024-09-17T11:41:58.205660+0000","last_unstale":"2024-09-17T11:42:26.427627+0000","last_undegraded":"2024-09-17T11:42:26.427627+0000","last_fullsized":"2024-09-17T11:42:26.427627+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:14.209247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814689+0000","last_change":"2024-09-17T11:41:55.982646+0000","last_active":"2024-09-17T11:42:26.814689+0000","last_peered":"2024-09-17T11:42:26.814689+0000","last_clean":"2024-09-17T11:42:26.814689+0000","last_became_active":"2024-09-17T11:41:55.982369+0000","last_became_peered":"2024-09-17T11:41:55.982369+0000","last_unstale":"2024-09-17T11:42:26.814689+0000","last_undegraded":"2024-09-17T11:42:26.814689+0000","last_fullsized":"2024-09-17T11:42:26.814689+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:53:10.743086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425854+0000","last_change":"2024-09-17T11:41:59.991592+0000","last_active":"2024-09-17T11:42:26.425854+0000","last_peered":"2024-09-17T11:42:26.425854+0000","last_clean":"2024-09-17T11:42:26.425854+0000","last_became_active":"2024-09-17T11:41:59.991318+0000","last_became_peered":"2024-09-17T11:41:59.991318+0000","last_unstale":"2024-09-17T11:42:26.425854+0000","last_undegraded":"2024-09-17T11:42:26.425854+0000","last_fullsized":"2024-09-17T11:42:26.425854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:58.569578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429549+0000","last_change":"2024-09-17T11:42:01.995676+0000","last_active":"2024-09-17T11:42:26.429549+0000","last_peered":"2024-09-17T11:42:26.429549+0000","last_clean":"2024-09-17T11:42:26.429549+0000","last_became_active":"2024-09-17T11:42:01.995369+0000","last_became_peered":"2024-09-17T11:42:01.995369+0000","last_unstale":"2024-09-17T11:42:26.429549+0000","last_undegraded":"2024-09-17T11:42:26.429549+0000","last_fullsized":"2024-09-17T11:42:26.429549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.040011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428824+0000","last_change":"2024-09-17T11:41:57.991836+0000","last_active":"2024-09-17T11:42:26.428824+0000","last_peered":"2024-09-17T11:42:26.428824+0000","last_clean":"2024-09-17T11:42:26.428824+0000","last_became_active":"2024-09-17T11:41:57.991462+0000","last_became_peered":"2024-09-17T11:41:57.991462+0000","last_unstale":"2024-09-17T11:42:26.428824+0000","last_undegraded":"2024-09-17T11:42:26.428824+0000","last_fullsized":"2024-09-17T11:42:26.428824+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:04:31.343026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426147+0000","last_change":"2024-09-17T11:41:55.988729+0000","last_active":"2024-09-17T11:42:26.426147+0000","last_peered":"2024-09-17T11:42:26.426147+0000","last_clean":"2024-09-17T11:42:26.426147+0000","last_became_active":"2024-09-17T11:41:55.987114+0000","last_became_peered":"2024-09-17T11:41:55.987114+0000","last_unstale":"2024-09-17T11:42:26.426147+0000","last_undegraded":"2024-09-17T11:42:26.426147+0000","last_fullsized":"2024-09-17T11:42:26.426147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:11.715778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815185+0000","last_change":"2024-09-17T11:41:59.991583+0000","last_active":"2024-09-17T11:42:26.815185+0000","last_peered":"2024-09-17T11:42:26.815185+0000","last_clean":"2024-09-17T11:42:26.815185+0000","last_became_active":"2024-09-17T11:41:59.991304+0000","last_became_peered":"2024-09-17T11:41:59.991304+0000","last_unstale":"2024-09-17T11:42:26.815185+0000","last_undegraded":"2024-09-17T11:42:26.815185+0000","last_fullsized":"2024-09-17T11:42:26.815185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:21:13.985381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426228+0000","last_change":"2024-09-17T11:42:02.004610+0000","last_active":"2024-09-17T11:42:26.426228+0000","last_peered":"2024-09-17T11:42:26.426228+0000","last_clean":"2024-09-17T11:42:26.426228+0000","last_became_active":"2024-09-17T11:42:02.003548+0000","last_became_peered":"2024-09-17T11:42:02.003548+0000","last_unstale":"2024-09-17T11:42:26.426228+0000","last_undegraded":"2024-09-17T11:42:26.426228+0000","last_fullsized":"2024-09-17T11:42:26.426228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:25:57.742990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814243+0000","last_change":"2024-09-17T11:42:02.003309+0000","last_active":"2024-09-17T11:42:26.814243+0000","last_peered":"2024-09-17T11:42:26.814243+0000","last_clean":"2024-09-17T11:42:26.814243+0000","last_became_active":"2024-09-17T11:42:02.002973+0000","last_became_peered":"2024-09-17T11:42:02.002973+0000","last_unstale":"2024-09-17T11:42:26.814243+0000","last_undegraded":"2024-09-17T11:42:26.814243+0000","last_fullsized":"2024-09-17T11:42:26.814243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:04.069209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429140+0000","last_change":"2024-09-17T11:41:55.981713+0000","last_active":"2024-09-17T11:42:26.429140+0000","last_peered":"2024-09-17T11:42:26.429140+0000","last_clean":"2024-09-17T11:42:26.429140+0000","last_became_active":"2024-09-17T11:41:55.981134+0000","last_became_peered":"2024-09-17T11:41:55.981134+0000","last_unstale":"2024-09-17T11:42:26.429140+0000","last_undegraded":"2024-09-17T11:42:26.429140+0000","last_fullsized":"2024-09-17T11:42:26.429140+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:30:55.599888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.816171+0000","last_change":"2024-09-17T11:41:57.986999+0000","last_active":"2024-09-17T11:42:26.816171+0000","last_peered":"2024-09-17T11:42:26.816171+0000","last_clean":"2024-09-17T11:42:26.816171+0000","last_became_active":"2024-09-17T11:41:57.986681+0000","last_became_peered":"2024-09-17T11:41:57.986681+0000","last_unstale":"2024-09-17T11:42:26.816171+0000","last_undegraded":"2024-09-17T11:42:26.816171+0000","last_fullsized":"2024-09-17T11:42:26.816171+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:03.860041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426610+0000","last_change":"2024-09-17T11:41:59.990661+0000","last_active":"2024-09-17T11:42:26.426610+0000","last_peered":"2024-09-17T11:42:26.426610+0000","last_clean":"2024-09-17T11:42:26.426610+0000","last_became_active":"2024-09-17T11:41:59.990341+0000","last_became_peered":"2024-09-17T11:41:59.990341+0000","last_unstale":"2024-09-17T11:42:26.426610+0000","last_undegraded":"2024-09-17T11:42:26.426610+0000","last_fullsized":"2024-09-17T11:42:26.426610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:21:49.240224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":31,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397515,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1232896,"data_stored":830670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":830578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789974,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713356,"statfs":{"total":95995035648,"available":95962476544,"internally_reserved":0,"allocated":622592,"data_stored":240556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986204,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31820,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713332,"statfs":{"total":95995035648,"available":95962451968,"internally_reserved":0,"allocated":647168,"data_stored":240245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149728,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717484,"statfs":{"total":95995035648,"available":95966703616,"internally_reserved":0,"allocated":655360,"data_stored":241088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411365,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31792,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713360,"statfs":{"total":95995035648,"available":95962480640,"internally_reserved":0,"allocated":618496,"data_stored":239801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717456,"statfs":{"total":95995035648,"available":95966674944,"internally_reserved":0,"allocated":684032,"data_stored":241425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1232896,"data_stored":830366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T11:42:41.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T11:42:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:41.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:41 smithi049 ceph-mon[30017]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:41.847 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:41 smithi049 ceph-mon[30017]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:41.847 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:41 smithi049 ceph-mon[40223]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:41.848 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:41 smithi049 ceph-mon[40223]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:41 smithi129 ceph-mon[33364]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:41 smithi129 ceph-mon[33364]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.230+0000 7ff94aa95640 1 -- 172.21.15.49:0/2847142446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944101c60 msgr2=0x7ff944102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.230+0000 7ff94aa95640 1 --2- 172.21.15.49:0/2847142446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944101c60 0x7ff944102040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff93802a410 comp rx=0 tx=0).stop 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 -- 172.21.15.49:0/2847142446 shutdown_connections 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 --2- 172.21.15.49:0/2847142446 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107ff0 0x7ff94410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 --2- 172.21.15.49:0/2847142446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944107670 0x7ff944107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 --2- 172.21.15.49:0/2847142446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944101c60 0x7ff944102040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 -- 172.21.15.49:0/2847142446 >> 172.21.15.49:0/2847142446 conn(0x7ff9440fd6d0 msgr2=0x7ff9440ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 -- 172.21.15.49:0/2847142446 shutdown_connections 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 -- 172.21.15.49:0/2847142446 wait complete. 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 Processor -- start 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.231+0000 7ff94aa95640 1 -- start start 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944101c60 0x7ff9441988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 0x7ff944198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 0x7ff94419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441124c0 con 0x7ff944107ff0 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff944112340 con 0x7ff944101c60 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff94aa95640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7ff944112640 con 0x7ff944107670 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff943fff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 0x7ff944198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 0x7ff94419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff943fff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 0x7ff944198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:39948/0 (socket says 172.21.15.49:39948) 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff943fff640 1 -- 172.21.15.49:0/3626953370 learned_addr learned my addr 172.21.15.49:0/3626953370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff948831640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944101c60 0x7ff9441988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 -- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 msgr2=0x7ff944198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 0x7ff944198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 -- 172.21.15.49:0/3626953370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944101c60 msgr2=0x7ff9441988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944101c60 0x7ff9441988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 -- 172.21.15.49:0/3626953370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94419e9a0 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff949032640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 0x7ff94419e1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ec50 tx=0x7ff93c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003b70 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff93c003d10 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.232+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c010650 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.233+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94419ec90 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.233+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441a5350 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.234+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff944102040 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.234+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7ff93c004020 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.235+0000 7ff941ffb640 1 --2- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 0x7ff93407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.235+0000 7ff948831640 1 --2- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 0x7ff93407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.235+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7ff93c09e430 con 0x7ff944107ff0 2024-09-17T11:42:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.235+0000 7ff948831640 1 --2- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 0x7ff93407d290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff938031440 tx=0x7ff938005b50 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.239+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff93c069c40 con 0x7ff944107ff0 2024-09-17T11:42:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.421+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 --> v2:172.21.15.49:6800/1163045599 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff944063590 con 0x7ff93407add0 2024-09-17T11:42:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.426+0000 7ff941ffb640 1 -- 172.21.15.49:0/3626953370 <== mgr.14619 v2:172.21.15.49:6800/1163045599 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346455 (secure 0 0 0) 0x7ff944063590 con 0x7ff93407add0 2024-09-17T11:42:42.432 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-09-17T11:42:42.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 msgr2=0x7ff93407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:42.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 0x7ff93407d290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff938031440 tx=0x7ff938005b50 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 msgr2=0x7ff94419e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 0x7ff94419e1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ec50 tx=0x7ff93c00c6a0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 shutdown_connections 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff93407add0 0x7ff93407d290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff944107ff0 0x7ff94419e1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff944107670 0x7ff944198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 --2- 172.21.15.49:0/3626953370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff944101c60 0x7ff9441988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 >> 172.21.15.49:0/3626953370 conn(0x7ff9440fd6d0 msgr2=0x7ff9440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 shutdown_connections 2024-09-17T11:42:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:42.430+0000 7ff94aa95640 1 -- 172.21.15.49:0/3626953370 wait complete. 2024-09-17T11:42:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:43.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:43.733 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-17T11:42:40.437485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":884,"num_read_kb":741,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":239600,"kb_used_data":6748,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749721616,"statfs":{"total":767960285184,"available":767714934784,"internally_reserved":0,"allocated":6909952,"data_stored":3694729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426066+0000","last_change":"2024-09-17T11:41:55.989582+0000","last_active":"2024-09-17T11:42:26.426066+0000","last_peered":"2024-09-17T11:42:26.426066+0000","last_clean":"2024-09-17T11:42:26.426066+0000","last_became_active":"2024-09-17T11:41:55.989141+0000","last_became_peered":"2024-09-17T11:41:55.989141+0000","last_unstale":"2024-09-17T11:42:26.426066+0000","last_undegraded":"2024-09-17T11:42:26.426066+0000","last_fullsized":"2024-09-17T11:42:26.426066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:12.831169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813707+0000","last_change":"2024-09-17T11:41:57.991084+0000","last_active":"2024-09-17T11:42:26.813707+0000","last_peered":"2024-09-17T11:42:26.813707+0000","last_clean":"2024-09-17T11:42:26.813707+0000","last_became_active":"2024-09-17T11:41:57.990552+0000","last_became_peered":"2024-09-17T11:41:57.990552+0000","last_unstale":"2024-09-17T11:42:26.813707+0000","last_undegraded":"2024-09-17T11:42:26.813707+0000","last_fullsized":"2024-09-17T11:42:26.813707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:11.988423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813731+0000","last_change":"2024-09-17T11:41:59.993877+0000","last_active":"2024-09-17T11:42:26.813731+0000","last_peered":"2024-09-17T11:42:26.813731+0000","last_clean":"2024-09-17T11:42:26.813731+0000","last_became_active":"2024-09-17T11:41:59.993555+0000","last_became_peered":"2024-09-17T11:41:59.993555+0000","last_unstale":"2024-09-17T11:42:26.813731+0000","last_undegraded":"2024-09-17T11:42:26.813731+0000","last_fullsized":"2024-09-17T11:42:26.813731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:24.333635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814115+0000","last_change":"2024-09-17T11:42:02.003670+0000","last_active":"2024-09-17T11:42:26.814115+0000","last_peered":"2024-09-17T11:42:26.814115+0000","last_clean":"2024-09-17T11:42:26.814115+0000","last_became_active":"2024-09-17T11:42:02.002935+0000","last_became_peered":"2024-09-17T11:42:02.002935+0000","last_unstale":"2024-09-17T11:42:26.814115+0000","last_undegraded":"2024-09-17T11:42:26.814115+0000","last_fullsized":"2024-09-17T11:42:26.814115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:51.700741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427727+0000","last_change":"2024-09-17T11:42:02.000886+0000","last_active":"2024-09-17T11:42:26.427727+0000","last_peered":"2024-09-17T11:42:26.427727+0000","last_clean":"2024-09-17T11:42:26.427727+0000","last_became_active":"2024-09-17T11:42:01.999941+0000","last_became_peered":"2024-09-17T11:42:01.999941+0000","last_unstale":"2024-09-17T11:42:26.427727+0000","last_undegraded":"2024-09-17T11:42:26.427727+0000","last_fullsized":"2024-09-17T11:42:26.427727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:35:02.189227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429628+0000","last_change":"2024-09-17T11:41:55.974362+0000","last_active":"2024-09-17T11:42:26.429628+0000","last_peered":"2024-09-17T11:42:26.429628+0000","last_clean":"2024-09-17T11:42:26.429628+0000","last_became_active":"2024-09-17T11:41:55.974036+0000","last_became_peered":"2024-09-17T11:41:55.974036+0000","last_unstale":"2024-09-17T11:42:26.429628+0000","last_undegraded":"2024-09-17T11:42:26.429628+0000","last_fullsized":"2024-09-17T11:42:26.429628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:07:21.608904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427933+0000","last_change":"2024-09-17T11:41:57.986171+0000","last_active":"2024-09-17T11:42:26.427933+0000","last_peered":"2024-09-17T11:42:26.427933+0000","last_clean":"2024-09-17T11:42:26.427933+0000","last_became_active":"2024-09-17T11:41:57.985808+0000","last_became_peered":"2024-09-17T11:41:57.985808+0000","last_unstale":"2024-09-17T11:42:26.427933+0000","last_undegraded":"2024-09-17T11:42:26.427933+0000","last_fullsized":"2024-09-17T11:42:26.427933+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:50:58.594905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813644+0000","last_change":"2024-09-17T11:41:59.994549+0000","last_active":"2024-09-17T11:42:26.813644+0000","last_peered":"2024-09-17T11:42:26.813644+0000","last_clean":"2024-09-17T11:42:26.813644+0000","last_became_active":"2024-09-17T11:41:59.994247+0000","last_became_peered":"2024-09-17T11:41:59.994247+0000","last_unstale":"2024-09-17T11:42:26.813644+0000","last_undegraded":"2024-09-17T11:42:26.813644+0000","last_fullsized":"2024-09-17T11:42:26.813644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:29:15.122502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815219+0000","last_change":"2024-09-17T11:41:55.981740+0000","last_active":"2024-09-17T11:42:26.815219+0000","last_peered":"2024-09-17T11:42:26.815219+0000","last_clean":"2024-09-17T11:42:26.815219+0000","last_became_active":"2024-09-17T11:41:55.981484+0000","last_became_peered":"2024-09-17T11:41:55.981484+0000","last_unstale":"2024-09-17T11:42:26.815219+0000","last_undegraded":"2024-09-17T11:42:26.815219+0000","last_fullsized":"2024-09-17T11:42:26.815219+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:18.526591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814513+0000","last_change":"2024-09-17T11:41:57.988080+0000","last_active":"2024-09-17T11:42:26.814513+0000","last_peered":"2024-09-17T11:42:26.814513+0000","last_clean":"2024-09-17T11:42:26.814513+0000","last_became_active":"2024-09-17T11:41:57.987836+0000","last_became_peered":"2024-09-17T11:41:57.987836+0000","last_unstale":"2024-09-17T11:42:26.814513+0000","last_undegraded":"2024-09-17T11:42:26.814513+0000","last_fullsized":"2024-09-17T11:42:26.814513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:58:02.180071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814368+0000","last_change":"2024-09-17T11:41:59.996325+0000","last_active":"2024-09-17T11:42:26.814368+0000","last_peered":"2024-09-17T11:42:26.814368+0000","last_clean":"2024-09-17T11:42:26.814368+0000","last_became_active":"2024-09-17T11:41:59.995335+0000","last_became_peered":"2024-09-17T11:41:59.995335+0000","last_unstale":"2024-09-17T11:42:26.814368+0000","last_undegraded":"2024-09-17T11:42:26.814368+0000","last_fullsized":"2024-09-17T11:42:26.814368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:02:53.382608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424763+0000","last_change":"2024-09-17T11:42:01.998933+0000","last_active":"2024-09-17T11:42:26.424763+0000","last_peered":"2024-09-17T11:42:26.424763+0000","last_clean":"2024-09-17T11:42:26.424763+0000","last_became_active":"2024-09-17T11:42:01.998619+0000","last_became_peered":"2024-09-17T11:42:01.998619+0000","last_unstale":"2024-09-17T11:42:26.424763+0000","last_undegraded":"2024-09-17T11:42:26.424763+0000","last_fullsized":"2024-09-17T11:42:26.424763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:53.371794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814693+0000","last_change":"2024-09-17T11:41:55.982408+0000","last_active":"2024-09-17T11:42:26.814693+0000","last_peered":"2024-09-17T11:42:26.814693+0000","last_clean":"2024-09-17T11:42:26.814693+0000","last_became_active":"2024-09-17T11:41:55.982121+0000","last_became_peered":"2024-09-17T11:41:55.982121+0000","last_unstale":"2024-09-17T11:42:26.814693+0000","last_undegraded":"2024-09-17T11:42:26.814693+0000","last_fullsized":"2024-09-17T11:42:26.814693+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:34:03.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'5","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814046+0000","last_change":"2024-09-17T11:41:57.990000+0000","last_active":"2024-09-17T11:42:26.814046+0000","last_peered":"2024-09-17T11:42:26.814046+0000","last_clean":"2024-09-17T11:42:26.814046+0000","last_became_active":"2024-09-17T11:41:57.989698+0000","last_became_peered":"2024-09-17T11:41:57.989698+0000","last_unstale":"2024-09-17T11:42:26.814046+0000","last_undegraded":"2024-09-17T11:42:26.814046+0000","last_fullsized":"2024-09-17T11:42:26.814046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:48.766237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426238+0000","last_change":"2024-09-17T11:41:59.997600+0000","last_active":"2024-09-17T11:42:26.426238+0000","last_peered":"2024-09-17T11:42:26.426238+0000","last_clean":"2024-09-17T11:42:26.426238+0000","last_became_active":"2024-09-17T11:41:59.997305+0000","last_became_peered":"2024-09-17T11:41:59.997305+0000","last_unstale":"2024-09-17T11:42:26.426238+0000","last_undegraded":"2024-09-17T11:42:26.426238+0000","last_fullsized":"2024-09-17T11:42:26.426238+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:23.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814010+0000","last_change":"2024-09-17T11:42:01.997327+0000","last_active":"2024-09-17T11:42:26.814010+0000","last_peered":"2024-09-17T11:42:26.814010+0000","last_clean":"2024-09-17T11:42:26.814010+0000","last_became_active":"2024-09-17T11:42:01.996978+0000","last_became_peered":"2024-09-17T11:42:01.996978+0000","last_unstale":"2024-09-17T11:42:26.814010+0000","last_undegraded":"2024-09-17T11:42:26.814010+0000","last_fullsized":"2024-09-17T11:42:26.814010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:55:26.728237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814071+0000","last_change":"2024-09-17T11:42:02.003899+0000","last_active":"2024-09-17T11:42:26.814071+0000","last_peered":"2024-09-17T11:42:26.814071+0000","last_clean":"2024-09-17T11:42:26.814071+0000","last_became_active":"2024-09-17T11:42:02.003579+0000","last_became_peered":"2024-09-17T11:42:02.003579+0000","last_unstale":"2024-09-17T11:42:26.814071+0000","last_undegraded":"2024-09-17T11:42:26.814071+0000","last_fullsized":"2024-09-17T11:42:26.814071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:55.341047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425837+0000","last_change":"2024-09-17T11:41:55.986976+0000","last_active":"2024-09-17T11:42:26.425837+0000","last_peered":"2024-09-17T11:42:26.425837+0000","last_clean":"2024-09-17T11:42:26.425837+0000","last_became_active":"2024-09-17T11:41:55.986667+0000","last_became_peered":"2024-09-17T11:41:55.986667+0000","last_unstale":"2024-09-17T11:42:26.425837+0000","last_undegraded":"2024-09-17T11:42:26.425837+0000","last_fullsized":"2024-09-17T11:42:26.425837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:48.330375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814766+0000","last_change":"2024-09-17T11:41:57.989989+0000","last_active":"2024-09-17T11:42:26.814766+0000","last_peered":"2024-09-17T11:42:26.814766+0000","last_clean":"2024-09-17T11:42:26.814766+0000","last_became_active":"2024-09-17T11:41:57.989671+0000","last_became_peered":"2024-09-17T11:41:57.989671+0000","last_unstale":"2024-09-17T11:42:26.814766+0000","last_undegraded":"2024-09-17T11:42:26.814766+0000","last_fullsized":"2024-09-17T11:42:26.814766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.169198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814540+0000","last_change":"2024-09-17T11:41:59.993889+0000","last_active":"2024-09-17T11:42:26.814540+0000","last_peered":"2024-09-17T11:42:26.814540+0000","last_clean":"2024-09-17T11:42:26.814540+0000","last_became_active":"2024-09-17T11:41:59.993562+0000","last_became_peered":"2024-09-17T11:41:59.993562+0000","last_unstale":"2024-09-17T11:42:26.814540+0000","last_undegraded":"2024-09-17T11:42:26.814540+0000","last_fullsized":"2024-09-17T11:42:26.814540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:14:59.117798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428794+0000","last_change":"2024-09-17T11:42:02.003394+0000","last_active":"2024-09-17T11:42:26.428794+0000","last_peered":"2024-09-17T11:42:26.428794+0000","last_clean":"2024-09-17T11:42:26.428794+0000","last_became_active":"2024-09-17T11:42:02.003094+0000","last_became_peered":"2024-09-17T11:42:02.003094+0000","last_unstale":"2024-09-17T11:42:26.428794+0000","last_undegraded":"2024-09-17T11:42:26.428794+0000","last_fullsized":"2024-09-17T11:42:26.428794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:51:07.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814692+0000","last_change":"2024-09-17T11:41:55.987999+0000","last_active":"2024-09-17T11:42:26.814692+0000","last_peered":"2024-09-17T11:42:26.814692+0000","last_clean":"2024-09-17T11:42:26.814692+0000","last_became_active":"2024-09-17T11:41:55.987725+0000","last_became_peered":"2024-09-17T11:41:55.987725+0000","last_unstale":"2024-09-17T11:42:26.814692+0000","last_undegraded":"2024-09-17T11:42:26.814692+0000","last_fullsized":"2024-09-17T11:42:26.814692+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:17:10.141698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428002+0000","last_change":"2024-09-17T11:41:57.989986+0000","last_active":"2024-09-17T11:42:26.428002+0000","last_peered":"2024-09-17T11:42:26.428002+0000","last_clean":"2024-09-17T11:42:26.428002+0000","last_became_active":"2024-09-17T11:41:57.989698+0000","last_became_peered":"2024-09-17T11:41:57.989698+0000","last_unstale":"2024-09-17T11:42:26.428002+0000","last_undegraded":"2024-09-17T11:42:26.428002+0000","last_fullsized":"2024-09-17T11:42:26.428002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:11.676870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814196+0000","last_change":"2024-09-17T11:41:59.984949+0000","last_active":"2024-09-17T11:42:26.814196+0000","last_peered":"2024-09-17T11:42:26.814196+0000","last_clean":"2024-09-17T11:42:26.814196+0000","last_became_active":"2024-09-17T11:41:59.984678+0000","last_became_peered":"2024-09-17T11:41:59.984678+0000","last_unstale":"2024-09-17T11:42:26.814196+0000","last_undegraded":"2024-09-17T11:42:26.814196+0000","last_fullsized":"2024-09-17T11:42:26.814196+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:10.056996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426497+0000","last_change":"2024-09-17T11:42:02.004483+0000","last_active":"2024-09-17T11:42:26.426497+0000","last_peered":"2024-09-17T11:42:26.426497+0000","last_clean":"2024-09-17T11:42:26.426497+0000","last_became_active":"2024-09-17T11:42:02.004055+0000","last_became_peered":"2024-09-17T11:42:02.004055+0000","last_unstale":"2024-09-17T11:42:26.426497+0000","last_undegraded":"2024-09-17T11:42:26.426497+0000","last_fullsized":"2024-09-17T11:42:26.426497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:34.942282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813865+0000","last_change":"2024-09-17T11:41:55.982105+0000","last_active":"2024-09-17T11:42:26.813865+0000","last_peered":"2024-09-17T11:42:26.813865+0000","last_clean":"2024-09-17T11:42:26.813865+0000","last_became_active":"2024-09-17T11:41:55.981797+0000","last_became_peered":"2024-09-17T11:41:55.981797+0000","last_unstale":"2024-09-17T11:42:26.813865+0000","last_undegraded":"2024-09-17T11:42:26.813865+0000","last_fullsized":"2024-09-17T11:42:26.813865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:22:38.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426450+0000","last_change":"2024-09-17T11:41:58.208884+0000","last_active":"2024-09-17T11:42:26.426450+0000","last_peered":"2024-09-17T11:42:26.426450+0000","last_clean":"2024-09-17T11:42:26.426450+0000","last_became_active":"2024-09-17T11:41:58.208509+0000","last_became_peered":"2024-09-17T11:41:58.208509+0000","last_unstale":"2024-09-17T11:42:26.426450+0000","last_undegraded":"2024-09-17T11:42:26.426450+0000","last_fullsized":"2024-09-17T11:42:26.426450+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:35.337301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041127+0000","last_change":"2024-09-17T11:41:59.991917+0000","last_active":"2024-09-17T11:42:31.041127+0000","last_peered":"2024-09-17T11:42:31.041127+0000","last_clean":"2024-09-17T11:42:31.041127+0000","last_became_active":"2024-09-17T11:41:59.991342+0000","last_became_peered":"2024-09-17T11:41:59.991342+0000","last_unstale":"2024-09-17T11:42:31.041127+0000","last_undegraded":"2024-09-17T11:42:31.041127+0000","last_fullsized":"2024-09-17T11:42:31.041127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:28:28.808385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815106+0000","last_change":"2024-09-17T11:41:57.995355+0000","last_active":"2024-09-17T11:42:26.815106+0000","last_peered":"2024-09-17T11:42:26.815106+0000","last_clean":"2024-09-17T11:42:26.815106+0000","last_became_active":"2024-09-17T11:41:57.995045+0000","last_became_peered":"2024-09-17T11:41:57.995045+0000","last_unstale":"2024-09-17T11:42:26.815106+0000","last_undegraded":"2024-09-17T11:42:26.815106+0000","last_fullsized":"2024-09-17T11:42:26.815106+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:40:07.633894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429231+0000","last_change":"2024-09-17T11:41:55.983669+0000","last_active":"2024-09-17T11:42:26.429231+0000","last_peered":"2024-09-17T11:42:26.429231+0000","last_clean":"2024-09-17T11:42:26.429231+0000","last_became_active":"2024-09-17T11:41:55.983387+0000","last_became_peered":"2024-09-17T11:41:55.983387+0000","last_unstale":"2024-09-17T11:42:26.429231+0000","last_undegraded":"2024-09-17T11:42:26.429231+0000","last_fullsized":"2024-09-17T11:42:26.429231+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:24:15.164436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041485+0000","last_change":"2024-09-17T11:41:59.990356+0000","last_active":"2024-09-17T11:42:31.041485+0000","last_peered":"2024-09-17T11:42:31.041485+0000","last_clean":"2024-09-17T11:42:31.041485+0000","last_became_active":"2024-09-17T11:41:59.990042+0000","last_became_peered":"2024-09-17T11:41:59.990042+0000","last_unstale":"2024-09-17T11:42:31.041485+0000","last_undegraded":"2024-09-17T11:42:31.041485+0000","last_fullsized":"2024-09-17T11:42:31.041485+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:15:45.670580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814752+0000","last_change":"2024-09-17T11:42:02.006641+0000","last_active":"2024-09-17T11:42:26.814752+0000","last_peered":"2024-09-17T11:42:26.814752+0000","last_clean":"2024-09-17T11:42:26.814752+0000","last_became_active":"2024-09-17T11:42:02.006300+0000","last_became_peered":"2024-09-17T11:42:02.006300+0000","last_unstale":"2024-09-17T11:42:26.814752+0000","last_undegraded":"2024-09-17T11:42:26.814752+0000","last_fullsized":"2024-09-17T11:42:26.814752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:07.505904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"61'18","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428935+0000","last_change":"2024-09-17T11:41:58.206130+0000","last_active":"2024-09-17T11:42:26.428935+0000","last_peered":"2024-09-17T11:42:26.428935+0000","last_clean":"2024-09-17T11:42:26.428935+0000","last_became_active":"2024-09-17T11:41:58.205681+0000","last_became_peered":"2024-09-17T11:41:58.205681+0000","last_unstale":"2024-09-17T11:42:26.428935+0000","last_undegraded":"2024-09-17T11:42:26.428935+0000","last_fullsized":"2024-09-17T11:42:26.428935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:10.784028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429133+0000","last_change":"2024-09-17T11:41:55.984073+0000","last_active":"2024-09-17T11:42:26.429133+0000","last_peered":"2024-09-17T11:42:26.429133+0000","last_clean":"2024-09-17T11:42:26.429133+0000","last_became_active":"2024-09-17T11:41:55.983821+0000","last_became_peered":"2024-09-17T11:41:55.983821+0000","last_unstale":"2024-09-17T11:42:26.429133+0000","last_undegraded":"2024-09-17T11:42:26.429133+0000","last_fullsized":"2024-09-17T11:42:26.429133+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:28:14.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813742+0000","last_change":"2024-09-17T11:41:59.990838+0000","last_active":"2024-09-17T11:42:26.813742+0000","last_peered":"2024-09-17T11:42:26.813742+0000","last_clean":"2024-09-17T11:42:26.813742+0000","last_became_active":"2024-09-17T11:41:59.990549+0000","last_became_peered":"2024-09-17T11:41:59.990549+0000","last_unstale":"2024-09-17T11:42:26.813742+0000","last_undegraded":"2024-09-17T11:42:26.813742+0000","last_fullsized":"2024-09-17T11:42:26.813742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:42.605754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814343+0000","last_change":"2024-09-17T11:42:02.004328+0000","last_active":"2024-09-17T11:42:26.814343+0000","last_peered":"2024-09-17T11:42:26.814343+0000","last_clean":"2024-09-17T11:42:26.814343+0000","last_became_active":"2024-09-17T11:42:02.003378+0000","last_became_peered":"2024-09-17T11:42:02.003378+0000","last_unstale":"2024-09-17T11:42:26.814343+0000","last_undegraded":"2024-09-17T11:42:26.814343+0000","last_fullsized":"2024-09-17T11:42:26.814343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:47.129106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"61'14","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813834+0000","last_change":"2024-09-17T11:41:57.985302+0000","last_active":"2024-09-17T11:42:26.813834+0000","last_peered":"2024-09-17T11:42:26.813834+0000","last_clean":"2024-09-17T11:42:26.813834+0000","last_became_active":"2024-09-17T11:41:57.985005+0000","last_became_peered":"2024-09-17T11:41:57.985005+0000","last_unstale":"2024-09-17T11:42:26.813834+0000","last_undegraded":"2024-09-17T11:42:26.813834+0000","last_fullsized":"2024-09-17T11:42:26.813834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:13.712944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'1","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426721+0000","last_change":"2024-09-17T11:41:55.984977+0000","last_active":"2024-09-17T11:42:26.426721+0000","last_peered":"2024-09-17T11:42:26.426721+0000","last_clean":"2024-09-17T11:42:26.426721+0000","last_became_active":"2024-09-17T11:41:55.984696+0000","last_became_peered":"2024-09-17T11:41:55.984696+0000","last_unstale":"2024-09-17T11:42:26.426721+0000","last_undegraded":"2024-09-17T11:42:26.426721+0000","last_fullsized":"2024-09-17T11:42:26.426721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:06.534112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041185+0000","last_change":"2024-09-17T11:41:59.991503+0000","last_active":"2024-09-17T11:42:31.041185+0000","last_peered":"2024-09-17T11:42:31.041185+0000","last_clean":"2024-09-17T11:42:31.041185+0000","last_became_active":"2024-09-17T11:41:59.991159+0000","last_became_peered":"2024-09-17T11:41:59.991159+0000","last_unstale":"2024-09-17T11:42:31.041185+0000","last_undegraded":"2024-09-17T11:42:31.041185+0000","last_fullsized":"2024-09-17T11:42:31.041185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:56.635177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"61'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426410+0000","last_change":"2024-09-17T11:42:02.004031+0000","last_active":"2024-09-17T11:42:26.426410+0000","last_peered":"2024-09-17T11:42:26.426410+0000","last_clean":"2024-09-17T11:42:26.426410+0000","last_became_active":"2024-09-17T11:42:02.003732+0000","last_became_peered":"2024-09-17T11:42:02.003732+0000","last_unstale":"2024-09-17T11:42:26.426410+0000","last_undegraded":"2024-09-17T11:42:26.426410+0000","last_fullsized":"2024-09-17T11:42:26.426410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:29.575945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813961+0000","last_change":"2024-09-17T11:41:57.990483+0000","last_active":"2024-09-17T11:42:26.813961+0000","last_peered":"2024-09-17T11:42:26.813961+0000","last_clean":"2024-09-17T11:42:26.813961+0000","last_became_active":"2024-09-17T11:41:57.990131+0000","last_became_peered":"2024-09-17T11:41:57.990131+0000","last_unstale":"2024-09-17T11:42:26.813961+0000","last_undegraded":"2024-09-17T11:42:26.813961+0000","last_fullsized":"2024-09-17T11:42:26.813961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815297+0000","last_change":"2024-09-17T11:41:55.987143+0000","last_active":"2024-09-17T11:42:26.815297+0000","last_peered":"2024-09-17T11:42:26.815297+0000","last_clean":"2024-09-17T11:42:26.815297+0000","last_became_active":"2024-09-17T11:41:55.986842+0000","last_became_peered":"2024-09-17T11:41:55.986842+0000","last_unstale":"2024-09-17T11:42:26.815297+0000","last_undegraded":"2024-09-17T11:42:26.815297+0000","last_fullsized":"2024-09-17T11:42:26.815297+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:15.724779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041065+0000","last_change":"2024-09-17T11:41:59.996499+0000","last_active":"2024-09-17T11:42:31.041065+0000","last_peered":"2024-09-17T11:42:31.041065+0000","last_clean":"2024-09-17T11:42:31.041065+0000","last_became_active":"2024-09-17T11:41:59.996193+0000","last_became_peered":"2024-09-17T11:41:59.996193+0000","last_unstale":"2024-09-17T11:42:31.041065+0000","last_undegraded":"2024-09-17T11:42:31.041065+0000","last_fullsized":"2024-09-17T11:42:31.041065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:55.251678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424917+0000","last_change":"2024-09-17T11:42:01.996040+0000","last_active":"2024-09-17T11:42:26.424917+0000","last_peered":"2024-09-17T11:42:26.424917+0000","last_clean":"2024-09-17T11:42:26.424917+0000","last_became_active":"2024-09-17T11:42:01.995685+0000","last_became_peered":"2024-09-17T11:42:01.995685+0000","last_unstale":"2024-09-17T11:42:26.424917+0000","last_undegraded":"2024-09-17T11:42:26.424917+0000","last_fullsized":"2024-09-17T11:42:26.424917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:05.593119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"61'19","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425679+0000","last_change":"2024-09-17T11:41:58.208549+0000","last_active":"2024-09-17T11:42:26.425679+0000","last_peered":"2024-09-17T11:42:26.425679+0000","last_clean":"2024-09-17T11:42:26.425679+0000","last_became_active":"2024-09-17T11:41:58.208146+0000","last_became_peered":"2024-09-17T11:41:58.208146+0000","last_unstale":"2024-09-17T11:42:26.425679+0000","last_undegraded":"2024-09-17T11:42:26.425679+0000","last_fullsized":"2024-09-17T11:42:26.425679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:11:41.730484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814896+0000","last_change":"2024-09-17T11:41:55.986941+0000","last_active":"2024-09-17T11:42:26.814896+0000","last_peered":"2024-09-17T11:42:26.814896+0000","last_clean":"2024-09-17T11:42:26.814896+0000","last_became_active":"2024-09-17T11:41:55.986631+0000","last_became_peered":"2024-09-17T11:41:55.986631+0000","last_unstale":"2024-09-17T11:42:26.814896+0000","last_undegraded":"2024-09-17T11:42:26.814896+0000","last_fullsized":"2024-09-17T11:42:26.814896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:39.670685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815299+0000","last_change":"2024-09-17T11:41:59.994127+0000","last_active":"2024-09-17T11:42:26.815299+0000","last_peered":"2024-09-17T11:42:26.815299+0000","last_clean":"2024-09-17T11:42:26.815299+0000","last_became_active":"2024-09-17T11:41:59.993856+0000","last_became_peered":"2024-09-17T11:41:59.993856+0000","last_unstale":"2024-09-17T11:42:26.815299+0000","last_undegraded":"2024-09-17T11:42:26.815299+0000","last_fullsized":"2024-09-17T11:42:26.815299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:43.851796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813729+0000","last_change":"2024-09-17T11:42:01.990969+0000","last_active":"2024-09-17T11:42:26.813729+0000","last_peered":"2024-09-17T11:42:26.813729+0000","last_clean":"2024-09-17T11:42:26.813729+0000","last_became_active":"2024-09-17T11:42:01.990146+0000","last_became_peered":"2024-09-17T11:42:01.990146+0000","last_unstale":"2024-09-17T11:42:26.813729+0000","last_undegraded":"2024-09-17T11:42:26.813729+0000","last_fullsized":"2024-09-17T11:42:26.813729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:22.071049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"61'28","reported_seq":83,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041070+0000","last_change":"2024-09-17T11:41:57.990035+0000","last_active":"2024-09-17T11:42:31.041070+0000","last_peered":"2024-09-17T11:42:31.041070+0000","last_clean":"2024-09-17T11:42:31.041070+0000","last_became_active":"2024-09-17T11:41:57.989715+0000","last_became_peered":"2024-09-17T11:41:57.989715+0000","last_unstale":"2024-09-17T11:42:31.041070+0000","last_undegraded":"2024-09-17T11:42:31.041070+0000","last_fullsized":"2024-09-17T11:42:31.041070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:51.024351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814654+0000","last_change":"2024-09-17T11:41:55.985396+0000","last_active":"2024-09-17T11:42:26.814654+0000","last_peered":"2024-09-17T11:42:26.814654+0000","last_clean":"2024-09-17T11:42:26.814654+0000","last_became_active":"2024-09-17T11:41:55.984630+0000","last_became_peered":"2024-09-17T11:41:55.984630+0000","last_unstale":"2024-09-17T11:42:26.814654+0000","last_undegraded":"2024-09-17T11:42:26.814654+0000","last_fullsized":"2024-09-17T11:42:26.814654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:47:27.303630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815098+0000","last_change":"2024-09-17T11:41:57.983938+0000","last_active":"2024-09-17T11:42:26.815098+0000","last_peered":"2024-09-17T11:42:26.815098+0000","last_clean":"2024-09-17T11:42:26.815098+0000","last_became_active":"2024-09-17T11:41:54.975540+0000","last_became_peered":"2024-09-17T11:41:54.975540+0000","last_unstale":"2024-09-17T11:42:26.815098+0000","last_undegraded":"2024-09-17T11:42:26.815098+0000","last_fullsized":"2024-09-17T11:42:26.815098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:57.679692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022569360000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425792+0000","last_change":"2024-09-17T11:41:59.996303+0000","last_active":"2024-09-17T11:42:26.425792+0000","last_peered":"2024-09-17T11:42:26.425792+0000","last_clean":"2024-09-17T11:42:26.425792+0000","last_became_active":"2024-09-17T11:41:59.994936+0000","last_became_peered":"2024-09-17T11:41:59.994936+0000","last_unstale":"2024-09-17T11:42:26.425792+0000","last_undegraded":"2024-09-17T11:42:26.425792+0000","last_fullsized":"2024-09-17T11:42:26.425792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:18.022005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428293+0000","last_change":"2024-09-17T11:42:02.000936+0000","last_active":"2024-09-17T11:42:26.428293+0000","last_peered":"2024-09-17T11:42:26.428293+0000","last_clean":"2024-09-17T11:42:26.428293+0000","last_became_active":"2024-09-17T11:42:02.000592+0000","last_became_peered":"2024-09-17T11:42:02.000592+0000","last_unstale":"2024-09-17T11:42:26.428293+0000","last_undegraded":"2024-09-17T11:42:26.428293+0000","last_fullsized":"2024-09-17T11:42:26.428293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:39.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"61'13","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815136+0000","last_change":"2024-09-17T11:41:57.993050+0000","last_active":"2024-09-17T11:42:26.815136+0000","last_peered":"2024-09-17T11:42:26.815136+0000","last_clean":"2024-09-17T11:42:26.815136+0000","last_became_active":"2024-09-17T11:41:57.992660+0000","last_became_peered":"2024-09-17T11:41:57.992660+0000","last_unstale":"2024-09-17T11:42:26.815136+0000","last_undegraded":"2024-09-17T11:42:26.815136+0000","last_fullsized":"2024-09-17T11:42:26.815136+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:24:47.286666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813899+0000","last_change":"2024-09-17T11:41:55.987763+0000","last_active":"2024-09-17T11:42:26.813899+0000","last_peered":"2024-09-17T11:42:26.813899+0000","last_clean":"2024-09-17T11:42:26.813899+0000","last_became_active":"2024-09-17T11:41:55.987463+0000","last_became_peered":"2024-09-17T11:41:55.987463+0000","last_unstale":"2024-09-17T11:42:26.813899+0000","last_undegraded":"2024-09-17T11:42:26.813899+0000","last_fullsized":"2024-09-17T11:42:26.813899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:19.050443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815394+0000","last_change":"2024-09-17T11:41:57.977137+0000","last_active":"2024-09-17T11:42:26.815394+0000","last_peered":"2024-09-17T11:42:26.815394+0000","last_clean":"2024-09-17T11:42:26.815394+0000","last_became_active":"2024-09-17T11:41:54.977934+0000","last_became_peered":"2024-09-17T11:41:54.977934+0000","last_unstale":"2024-09-17T11:42:26.815394+0000","last_undegraded":"2024-09-17T11:42:26.815394+0000","last_fullsized":"2024-09-17T11:42:26.815394+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:42:38.476238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088804800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814665+0000","last_change":"2024-09-17T11:41:59.989964+0000","last_active":"2024-09-17T11:42:26.814665+0000","last_peered":"2024-09-17T11:42:26.814665+0000","last_clean":"2024-09-17T11:42:26.814665+0000","last_became_active":"2024-09-17T11:41:59.989650+0000","last_became_peered":"2024-09-17T11:41:59.989650+0000","last_unstale":"2024-09-17T11:42:26.814665+0000","last_undegraded":"2024-09-17T11:42:26.814665+0000","last_fullsized":"2024-09-17T11:42:26.814665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:35.426175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426534+0000","last_change":"2024-09-17T11:42:02.007937+0000","last_active":"2024-09-17T11:42:26.426534+0000","last_peered":"2024-09-17T11:42:26.426534+0000","last_clean":"2024-09-17T11:42:26.426534+0000","last_became_active":"2024-09-17T11:42:02.007385+0000","last_became_peered":"2024-09-17T11:42:02.007385+0000","last_unstale":"2024-09-17T11:42:26.426534+0000","last_undegraded":"2024-09-17T11:42:26.426534+0000","last_fullsized":"2024-09-17T11:42:26.426534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:01.632872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":44,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426580+0000","last_change":"2024-09-17T11:41:57.994215+0000","last_active":"2024-09-17T11:42:26.426580+0000","last_peered":"2024-09-17T11:42:26.426580+0000","last_clean":"2024-09-17T11:42:26.426580+0000","last_became_active":"2024-09-17T11:41:57.993899+0000","last_became_peered":"2024-09-17T11:41:57.993899+0000","last_unstale":"2024-09-17T11:42:26.426580+0000","last_undegraded":"2024-09-17T11:42:26.426580+0000","last_fullsized":"2024-09-17T11:42:26.426580+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:33:38.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425203+0000","last_change":"2024-09-17T11:41:55.985987+0000","last_active":"2024-09-17T11:42:26.425203+0000","last_peered":"2024-09-17T11:42:26.425203+0000","last_clean":"2024-09-17T11:42:26.425203+0000","last_became_active":"2024-09-17T11:41:55.985672+0000","last_became_peered":"2024-09-17T11:41:55.985672+0000","last_unstale":"2024-09-17T11:42:26.425203+0000","last_undegraded":"2024-09-17T11:42:26.425203+0000","last_fullsized":"2024-09-17T11:42:26.425203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:34:27.561847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:36.228216+0000","last_change":"2024-09-17T11:41:58.206699+0000","last_active":"2024-09-17T11:42:36.228216+0000","last_peered":"2024-09-17T11:42:36.228216+0000","last_clean":"2024-09-17T11:42:36.228216+0000","last_became_active":"2024-09-17T11:41:54.979092+0000","last_became_peered":"2024-09-17T11:41:54.979092+0000","last_unstale":"2024-09-17T11:42:36.228216+0000","last_undegraded":"2024-09-17T11:42:36.228216+0000","last_fullsized":"2024-09-17T11:42:36.228216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:53.961925+0000","last_clean_scrub_stamp":"2024-09-17T11:41:53.961925+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:33:49.383136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035626,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815545+0000","last_change":"2024-09-17T11:41:59.996698+0000","last_active":"2024-09-17T11:42:26.815545+0000","last_peered":"2024-09-17T11:42:26.815545+0000","last_clean":"2024-09-17T11:42:26.815545+0000","last_became_active":"2024-09-17T11:41:59.996421+0000","last_became_peered":"2024-09-17T11:41:59.996421+0000","last_unstale":"2024-09-17T11:42:26.815545+0000","last_undegraded":"2024-09-17T11:42:26.815545+0000","last_fullsized":"2024-09-17T11:42:26.815545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:44.901238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814440+0000","last_change":"2024-09-17T11:42:02.002018+0000","last_active":"2024-09-17T11:42:26.814440+0000","last_peered":"2024-09-17T11:42:26.814440+0000","last_clean":"2024-09-17T11:42:26.814440+0000","last_became_active":"2024-09-17T11:42:02.001686+0000","last_became_peered":"2024-09-17T11:42:02.001686+0000","last_unstale":"2024-09-17T11:42:26.814440+0000","last_undegraded":"2024-09-17T11:42:26.814440+0000","last_fullsized":"2024-09-17T11:42:26.814440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:15.445854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"61'16","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041191+0000","last_change":"2024-09-17T11:41:58.207627+0000","last_active":"2024-09-17T11:42:31.041191+0000","last_peered":"2024-09-17T11:42:31.041191+0000","last_clean":"2024-09-17T11:42:31.041191+0000","last_became_active":"2024-09-17T11:41:58.207249+0000","last_became_peered":"2024-09-17T11:41:58.207249+0000","last_unstale":"2024-09-17T11:42:31.041191+0000","last_undegraded":"2024-09-17T11:42:31.041191+0000","last_fullsized":"2024-09-17T11:42:31.041191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:42.670782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"54'1","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429440+0000","last_change":"2024-09-17T11:41:55.983112+0000","last_active":"2024-09-17T11:42:26.429440+0000","last_peered":"2024-09-17T11:42:26.429440+0000","last_clean":"2024-09-17T11:42:26.429440+0000","last_became_active":"2024-09-17T11:41:55.982889+0000","last_became_peered":"2024-09-17T11:41:55.982889+0000","last_unstale":"2024-09-17T11:42:26.429440+0000","last_undegraded":"2024-09-17T11:42:26.429440+0000","last_fullsized":"2024-09-17T11:42:26.429440+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:51:45.874787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"63'79","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:28.887229+0000","last_change":"2024-09-17T11:41:48.977527+0000","last_active":"2024-09-17T11:42:28.887229+0000","last_peered":"2024-09-17T11:42:28.887229+0000","last_clean":"2024-09-17T11:42:28.887229+0000","last_became_active":"2024-09-17T11:41:48.971851+0000","last_became_peered":"2024-09-17T11:41:48.971851+0000","last_unstale":"2024-09-17T11:42:28.887229+0000","last_undegraded":"2024-09-17T11:42:28.887229+0000","last_fullsized":"2024-09-17T11:42:28.887229+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:39:43.074460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:39:43.074460+0000","last_clean_scrub_stamp":"2024-09-17T11:39:43.074460+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:22.643693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426946+0000","last_change":"2024-09-17T11:41:59.990248+0000","last_active":"2024-09-17T11:42:26.426946+0000","last_peered":"2024-09-17T11:42:26.426946+0000","last_clean":"2024-09-17T11:42:26.426946+0000","last_became_active":"2024-09-17T11:41:59.989889+0000","last_became_peered":"2024-09-17T11:41:59.989889+0000","last_unstale":"2024-09-17T11:42:26.426946+0000","last_undegraded":"2024-09-17T11:42:26.426946+0000","last_fullsized":"2024-09-17T11:42:26.426946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:16.851610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815371+0000","last_change":"2024-09-17T11:42:02.002139+0000","last_active":"2024-09-17T11:42:26.815371+0000","last_peered":"2024-09-17T11:42:26.815371+0000","last_clean":"2024-09-17T11:42:26.815371+0000","last_became_active":"2024-09-17T11:42:02.001853+0000","last_became_peered":"2024-09-17T11:42:02.001853+0000","last_unstale":"2024-09-17T11:42:26.815371+0000","last_undegraded":"2024-09-17T11:42:26.815371+0000","last_fullsized":"2024-09-17T11:42:26.815371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:41:57.655567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814830+0000","last_change":"2024-09-17T11:41:57.989950+0000","last_active":"2024-09-17T11:42:26.814830+0000","last_peered":"2024-09-17T11:42:26.814830+0000","last_clean":"2024-09-17T11:42:26.814830+0000","last_became_active":"2024-09-17T11:41:57.989642+0000","last_became_peered":"2024-09-17T11:41:57.989642+0000","last_unstale":"2024-09-17T11:42:26.814830+0000","last_undegraded":"2024-09-17T11:42:26.814830+0000","last_fullsized":"2024-09-17T11:42:26.814830+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:01.799841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814295+0000","last_change":"2024-09-17T11:41:55.989222+0000","last_active":"2024-09-17T11:42:26.814295+0000","last_peered":"2024-09-17T11:42:26.814295+0000","last_clean":"2024-09-17T11:42:26.814295+0000","last_became_active":"2024-09-17T11:41:55.988674+0000","last_became_peered":"2024-09-17T11:41:55.988674+0000","last_unstale":"2024-09-17T11:42:26.814295+0000","last_undegraded":"2024-09-17T11:42:26.814295+0000","last_fullsized":"2024-09-17T11:42:26.814295+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:08.566186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815237+0000","last_change":"2024-09-17T11:41:59.988719+0000","last_active":"2024-09-17T11:42:26.815237+0000","last_peered":"2024-09-17T11:42:26.815237+0000","last_clean":"2024-09-17T11:42:26.815237+0000","last_became_active":"2024-09-17T11:41:59.988463+0000","last_became_peered":"2024-09-17T11:41:59.988463+0000","last_unstale":"2024-09-17T11:42:26.815237+0000","last_undegraded":"2024-09-17T11:42:26.815237+0000","last_fullsized":"2024-09-17T11:42:26.815237+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:50.655214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427799+0000","last_change":"2024-09-17T11:42:01.997964+0000","last_active":"2024-09-17T11:42:26.427799+0000","last_peered":"2024-09-17T11:42:26.427799+0000","last_clean":"2024-09-17T11:42:26.427799+0000","last_became_active":"2024-09-17T11:42:01.997672+0000","last_became_peered":"2024-09-17T11:42:01.997672+0000","last_unstale":"2024-09-17T11:42:26.427799+0000","last_undegraded":"2024-09-17T11:42:26.427799+0000","last_fullsized":"2024-09-17T11:42:26.427799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:20:42.353045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814498+0000","last_change":"2024-09-17T11:41:57.991635+0000","last_active":"2024-09-17T11:42:26.814498+0000","last_peered":"2024-09-17T11:42:26.814498+0000","last_clean":"2024-09-17T11:42:26.814498+0000","last_became_active":"2024-09-17T11:41:57.990969+0000","last_became_peered":"2024-09-17T11:41:57.990969+0000","last_unstale":"2024-09-17T11:42:26.814498+0000","last_undegraded":"2024-09-17T11:42:26.814498+0000","last_fullsized":"2024-09-17T11:42:26.814498+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:36.826155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815528+0000","last_change":"2024-09-17T11:41:55.979515+0000","last_active":"2024-09-17T11:42:26.815528+0000","last_peered":"2024-09-17T11:42:26.815528+0000","last_clean":"2024-09-17T11:42:26.815528+0000","last_became_active":"2024-09-17T11:41:55.979188+0000","last_became_peered":"2024-09-17T11:41:55.979188+0000","last_unstale":"2024-09-17T11:42:26.815528+0000","last_undegraded":"2024-09-17T11:42:26.815528+0000","last_fullsized":"2024-09-17T11:42:26.815528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:41:15.514646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814543+0000","last_change":"2024-09-17T11:41:59.998804+0000","last_active":"2024-09-17T11:42:26.814543+0000","last_peered":"2024-09-17T11:42:26.814543+0000","last_clean":"2024-09-17T11:42:26.814543+0000","last_became_active":"2024-09-17T11:41:59.998488+0000","last_became_peered":"2024-09-17T11:41:59.998488+0000","last_unstale":"2024-09-17T11:42:26.814543+0000","last_undegraded":"2024-09-17T11:42:26.814543+0000","last_fullsized":"2024-09-17T11:42:26.814543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:22:37.439485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814561+0000","last_change":"2024-09-17T11:42:01.998759+0000","last_active":"2024-09-17T11:42:26.814561+0000","last_peered":"2024-09-17T11:42:26.814561+0000","last_clean":"2024-09-17T11:42:26.814561+0000","last_became_active":"2024-09-17T11:42:01.998446+0000","last_became_peered":"2024-09-17T11:42:01.998446+0000","last_unstale":"2024-09-17T11:42:26.814561+0000","last_undegraded":"2024-09-17T11:42:26.814561+0000","last_fullsized":"2024-09-17T11:42:26.814561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:41.175648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814306+0000","last_change":"2024-09-17T11:41:57.987267+0000","last_active":"2024-09-17T11:42:26.814306+0000","last_peered":"2024-09-17T11:42:26.814306+0000","last_clean":"2024-09-17T11:42:26.814306+0000","last_became_active":"2024-09-17T11:41:57.986949+0000","last_became_peered":"2024-09-17T11:41:57.986949+0000","last_unstale":"2024-09-17T11:42:26.814306+0000","last_undegraded":"2024-09-17T11:42:26.814306+0000","last_fullsized":"2024-09-17T11:42:26.814306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:03.940212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429643+0000","last_change":"2024-09-17T11:41:55.983110+0000","last_active":"2024-09-17T11:42:26.429643+0000","last_peered":"2024-09-17T11:42:26.429643+0000","last_clean":"2024-09-17T11:42:26.429643+0000","last_became_active":"2024-09-17T11:41:55.982829+0000","last_became_peered":"2024-09-17T11:41:55.982829+0000","last_unstale":"2024-09-17T11:42:26.429643+0000","last_undegraded":"2024-09-17T11:42:26.429643+0000","last_fullsized":"2024-09-17T11:42:26.429643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:51.255636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041051+0000","last_change":"2024-09-17T11:41:59.990588+0000","last_active":"2024-09-17T11:42:31.041051+0000","last_peered":"2024-09-17T11:42:31.041051+0000","last_clean":"2024-09-17T11:42:31.041051+0000","last_became_active":"2024-09-17T11:41:59.990273+0000","last_became_peered":"2024-09-17T11:41:59.990273+0000","last_unstale":"2024-09-17T11:42:31.041051+0000","last_undegraded":"2024-09-17T11:42:31.041051+0000","last_fullsized":"2024-09-17T11:42:31.041051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:29:02.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813984+0000","last_change":"2024-09-17T11:42:02.003543+0000","last_active":"2024-09-17T11:42:26.813984+0000","last_peered":"2024-09-17T11:42:26.813984+0000","last_clean":"2024-09-17T11:42:26.813984+0000","last_became_active":"2024-09-17T11:42:02.003264+0000","last_became_peered":"2024-09-17T11:42:02.003264+0000","last_unstale":"2024-09-17T11:42:26.813984+0000","last_undegraded":"2024-09-17T11:42:26.813984+0000","last_fullsized":"2024-09-17T11:42:26.813984+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:14:04.659732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425578+0000","last_change":"2024-09-17T11:41:57.990006+0000","last_active":"2024-09-17T11:42:26.425578+0000","last_peered":"2024-09-17T11:42:26.425578+0000","last_clean":"2024-09-17T11:42:26.425578+0000","last_became_active":"2024-09-17T11:41:57.989716+0000","last_became_peered":"2024-09-17T11:41:57.989716+0000","last_unstale":"2024-09-17T11:42:26.425578+0000","last_undegraded":"2024-09-17T11:42:26.425578+0000","last_fullsized":"2024-09-17T11:42:26.425578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:07.100432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814796+0000","last_change":"2024-09-17T11:41:55.979389+0000","last_active":"2024-09-17T11:42:26.814796+0000","last_peered":"2024-09-17T11:42:26.814796+0000","last_clean":"2024-09-17T11:42:26.814796+0000","last_became_active":"2024-09-17T11:41:55.979072+0000","last_became_peered":"2024-09-17T11:41:55.979072+0000","last_unstale":"2024-09-17T11:42:26.814796+0000","last_undegraded":"2024-09-17T11:42:26.814796+0000","last_fullsized":"2024-09-17T11:42:26.814796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:35:27.267245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815366+0000","last_change":"2024-09-17T11:41:59.985802+0000","last_active":"2024-09-17T11:42:26.815366+0000","last_peered":"2024-09-17T11:42:26.815366+0000","last_clean":"2024-09-17T11:42:26.815366+0000","last_became_active":"2024-09-17T11:41:59.985285+0000","last_became_peered":"2024-09-17T11:41:59.985285+0000","last_unstale":"2024-09-17T11:42:26.815366+0000","last_undegraded":"2024-09-17T11:42:26.815366+0000","last_fullsized":"2024-09-17T11:42:26.815366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:38.527214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425106+0000","last_change":"2024-09-17T11:42:01.998275+0000","last_active":"2024-09-17T11:42:26.425106+0000","last_peered":"2024-09-17T11:42:26.425106+0000","last_clean":"2024-09-17T11:42:26.425106+0000","last_became_active":"2024-09-17T11:42:01.997994+0000","last_became_peered":"2024-09-17T11:42:01.997994+0000","last_unstale":"2024-09-17T11:42:26.425106+0000","last_undegraded":"2024-09-17T11:42:26.425106+0000","last_fullsized":"2024-09-17T11:42:26.425106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:54:09.619950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"61'19","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815365+0000","last_change":"2024-09-17T11:41:58.208112+0000","last_active":"2024-09-17T11:42:26.815365+0000","last_peered":"2024-09-17T11:42:26.815365+0000","last_clean":"2024-09-17T11:42:26.815365+0000","last_became_active":"2024-09-17T11:41:58.207290+0000","last_became_peered":"2024-09-17T11:41:58.207290+0000","last_unstale":"2024-09-17T11:42:26.815365+0000","last_undegraded":"2024-09-17T11:42:26.815365+0000","last_fullsized":"2024-09-17T11:42:26.815365+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:26.241882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426940+0000","last_change":"2024-09-17T11:41:55.985115+0000","last_active":"2024-09-17T11:42:26.426940+0000","last_peered":"2024-09-17T11:42:26.426940+0000","last_clean":"2024-09-17T11:42:26.426940+0000","last_became_active":"2024-09-17T11:41:55.984842+0000","last_became_peered":"2024-09-17T11:41:55.984842+0000","last_unstale":"2024-09-17T11:42:26.426940+0000","last_undegraded":"2024-09-17T11:42:26.426940+0000","last_fullsized":"2024-09-17T11:42:26.426940+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:50.329355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814042+0000","last_change":"2024-09-17T11:41:59.989276+0000","last_active":"2024-09-17T11:42:26.814042+0000","last_peered":"2024-09-17T11:42:26.814042+0000","last_clean":"2024-09-17T11:42:26.814042+0000","last_became_active":"2024-09-17T11:41:59.988964+0000","last_became_peered":"2024-09-17T11:41:59.988964+0000","last_unstale":"2024-09-17T11:42:26.814042+0000","last_undegraded":"2024-09-17T11:42:26.814042+0000","last_fullsized":"2024-09-17T11:42:26.814042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:36.955466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426216+0000","last_change":"2024-09-17T11:42:02.005488+0000","last_active":"2024-09-17T11:42:26.426216+0000","last_peered":"2024-09-17T11:42:26.426216+0000","last_clean":"2024-09-17T11:42:26.426216+0000","last_became_active":"2024-09-17T11:42:02.005175+0000","last_became_peered":"2024-09-17T11:42:02.005175+0000","last_unstale":"2024-09-17T11:42:26.426216+0000","last_undegraded":"2024-09-17T11:42:26.426216+0000","last_fullsized":"2024-09-17T11:42:26.426216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:32.740868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'12","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814159+0000","last_change":"2024-09-17T11:41:57.986300+0000","last_active":"2024-09-17T11:42:26.814159+0000","last_peered":"2024-09-17T11:42:26.814159+0000","last_clean":"2024-09-17T11:42:26.814159+0000","last_became_active":"2024-09-17T11:41:57.985853+0000","last_became_peered":"2024-09-17T11:41:57.985853+0000","last_unstale":"2024-09-17T11:42:26.814159+0000","last_undegraded":"2024-09-17T11:42:26.814159+0000","last_fullsized":"2024-09-17T11:42:26.814159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:54:54.946518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426980+0000","last_change":"2024-09-17T11:41:55.990216+0000","last_active":"2024-09-17T11:42:26.426980+0000","last_peered":"2024-09-17T11:42:26.426980+0000","last_clean":"2024-09-17T11:42:26.426980+0000","last_became_active":"2024-09-17T11:41:55.989967+0000","last_became_peered":"2024-09-17T11:41:55.989967+0000","last_unstale":"2024-09-17T11:42:26.426980+0000","last_undegraded":"2024-09-17T11:42:26.426980+0000","last_fullsized":"2024-09-17T11:42:26.426980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:34.401540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814409+0000","last_change":"2024-09-17T11:41:59.995275+0000","last_active":"2024-09-17T11:42:26.814409+0000","last_peered":"2024-09-17T11:42:26.814409+0000","last_clean":"2024-09-17T11:42:26.814409+0000","last_became_active":"2024-09-17T11:41:59.994712+0000","last_became_peered":"2024-09-17T11:41:59.994712+0000","last_unstale":"2024-09-17T11:42:26.814409+0000","last_undegraded":"2024-09-17T11:42:26.814409+0000","last_fullsized":"2024-09-17T11:42:26.814409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:21.580649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429270+0000","last_change":"2024-09-17T11:42:02.001168+0000","last_active":"2024-09-17T11:42:26.429270+0000","last_peered":"2024-09-17T11:42:26.429270+0000","last_clean":"2024-09-17T11:42:26.429270+0000","last_became_active":"2024-09-17T11:42:02.000257+0000","last_became_peered":"2024-09-17T11:42:02.000257+0000","last_unstale":"2024-09-17T11:42:26.429270+0000","last_undegraded":"2024-09-17T11:42:26.429270+0000","last_fullsized":"2024-09-17T11:42:26.429270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:43.861697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"61'15","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813617+0000","last_change":"2024-09-17T11:41:58.206453+0000","last_active":"2024-09-17T11:42:26.813617+0000","last_peered":"2024-09-17T11:42:26.813617+0000","last_clean":"2024-09-17T11:42:26.813617+0000","last_became_active":"2024-09-17T11:41:58.206166+0000","last_became_peered":"2024-09-17T11:41:58.206166+0000","last_unstale":"2024-09-17T11:42:26.813617+0000","last_undegraded":"2024-09-17T11:42:26.813617+0000","last_fullsized":"2024-09-17T11:42:26.813617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:47.145321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426828+0000","last_change":"2024-09-17T11:41:55.988303+0000","last_active":"2024-09-17T11:42:26.426828+0000","last_peered":"2024-09-17T11:42:26.426828+0000","last_clean":"2024-09-17T11:42:26.426828+0000","last_became_active":"2024-09-17T11:41:55.987994+0000","last_became_peered":"2024-09-17T11:41:55.987994+0000","last_unstale":"2024-09-17T11:42:26.426828+0000","last_undegraded":"2024-09-17T11:42:26.426828+0000","last_fullsized":"2024-09-17T11:42:26.426828+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:37.381057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041129+0000","last_change":"2024-09-17T11:41:59.992449+0000","last_active":"2024-09-17T11:42:31.041129+0000","last_peered":"2024-09-17T11:42:31.041129+0000","last_clean":"2024-09-17T11:42:31.041129+0000","last_became_active":"2024-09-17T11:41:59.992128+0000","last_became_peered":"2024-09-17T11:41:59.992128+0000","last_unstale":"2024-09-17T11:42:31.041129+0000","last_undegraded":"2024-09-17T11:42:31.041129+0000","last_fullsized":"2024-09-17T11:42:31.041129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:06.370600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813917+0000","last_change":"2024-09-17T11:42:01.997709+0000","last_active":"2024-09-17T11:42:26.813917+0000","last_peered":"2024-09-17T11:42:26.813917+0000","last_clean":"2024-09-17T11:42:26.813917+0000","last_became_active":"2024-09-17T11:42:01.997004+0000","last_became_peered":"2024-09-17T11:42:01.997004+0000","last_unstale":"2024-09-17T11:42:26.813917+0000","last_undegraded":"2024-09-17T11:42:26.813917+0000","last_fullsized":"2024-09-17T11:42:26.813917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:22.940365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813762+0000","last_change":"2024-09-17T11:41:55.980616+0000","last_active":"2024-09-17T11:42:26.813762+0000","last_peered":"2024-09-17T11:42:26.813762+0000","last_clean":"2024-09-17T11:42:26.813762+0000","last_became_active":"2024-09-17T11:41:55.980349+0000","last_became_peered":"2024-09-17T11:41:55.980349+0000","last_unstale":"2024-09-17T11:42:26.813762+0000","last_undegraded":"2024-09-17T11:42:26.813762+0000","last_fullsized":"2024-09-17T11:42:26.813762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:08:04.927608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'6","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428994+0000","last_change":"2024-09-17T11:41:58.207530+0000","last_active":"2024-09-17T11:42:26.428994+0000","last_peered":"2024-09-17T11:42:26.428994+0000","last_clean":"2024-09-17T11:42:26.428994+0000","last_became_active":"2024-09-17T11:41:58.207152+0000","last_became_peered":"2024-09-17T11:41:58.207152+0000","last_unstale":"2024-09-17T11:42:26.428994+0000","last_undegraded":"2024-09-17T11:42:26.428994+0000","last_fullsized":"2024-09-17T11:42:26.428994+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:08.737262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814598+0000","last_change":"2024-09-17T11:41:59.991365+0000","last_active":"2024-09-17T11:42:26.814598+0000","last_peered":"2024-09-17T11:42:26.814598+0000","last_clean":"2024-09-17T11:42:26.814598+0000","last_became_active":"2024-09-17T11:41:59.991058+0000","last_became_peered":"2024-09-17T11:41:59.991058+0000","last_unstale":"2024-09-17T11:42:26.814598+0000","last_undegraded":"2024-09-17T11:42:26.814598+0000","last_fullsized":"2024-09-17T11:42:26.814598+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:21:02.626705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426454+0000","last_change":"2024-09-17T11:42:02.001290+0000","last_active":"2024-09-17T11:42:26.426454+0000","last_peered":"2024-09-17T11:42:26.426454+0000","last_clean":"2024-09-17T11:42:26.426454+0000","last_became_active":"2024-09-17T11:42:02.000165+0000","last_became_peered":"2024-09-17T11:42:02.000165+0000","last_unstale":"2024-09-17T11:42:26.426454+0000","last_undegraded":"2024-09-17T11:42:26.426454+0000","last_fullsized":"2024-09-17T11:42:26.426454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:53.699106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426699+0000","last_change":"2024-09-17T11:41:58.209918+0000","last_active":"2024-09-17T11:42:26.426699+0000","last_peered":"2024-09-17T11:42:26.426699+0000","last_clean":"2024-09-17T11:42:26.426699+0000","last_became_active":"2024-09-17T11:41:58.209531+0000","last_became_peered":"2024-09-17T11:41:58.209531+0000","last_unstale":"2024-09-17T11:42:26.426699+0000","last_undegraded":"2024-09-17T11:42:26.426699+0000","last_fullsized":"2024-09-17T11:42:26.426699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:27.508359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426646+0000","last_change":"2024-09-17T11:41:55.989087+0000","last_active":"2024-09-17T11:42:26.426646+0000","last_peered":"2024-09-17T11:42:26.426646+0000","last_clean":"2024-09-17T11:42:26.426646+0000","last_became_active":"2024-09-17T11:41:55.988807+0000","last_became_peered":"2024-09-17T11:41:55.988807+0000","last_unstale":"2024-09-17T11:42:26.426646+0000","last_undegraded":"2024-09-17T11:42:26.426646+0000","last_fullsized":"2024-09-17T11:42:26.426646+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:42.340712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428870+0000","last_change":"2024-09-17T11:41:59.995712+0000","last_active":"2024-09-17T11:42:26.428870+0000","last_peered":"2024-09-17T11:42:26.428870+0000","last_clean":"2024-09-17T11:42:26.428870+0000","last_became_active":"2024-09-17T11:41:59.995390+0000","last_became_peered":"2024-09-17T11:41:59.995390+0000","last_unstale":"2024-09-17T11:42:26.428870+0000","last_undegraded":"2024-09-17T11:42:26.428870+0000","last_fullsized":"2024-09-17T11:42:26.428870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:30:45.824979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814112+0000","last_change":"2024-09-17T11:42:01.999831+0000","last_active":"2024-09-17T11:42:26.814112+0000","last_peered":"2024-09-17T11:42:26.814112+0000","last_clean":"2024-09-17T11:42:26.814112+0000","last_became_active":"2024-09-17T11:42:01.998381+0000","last_became_peered":"2024-09-17T11:42:01.998381+0000","last_unstale":"2024-09-17T11:42:26.814112+0000","last_undegraded":"2024-09-17T11:42:26.814112+0000","last_fullsized":"2024-09-17T11:42:26.814112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:07.503273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815097+0000","last_change":"2024-09-17T11:41:58.208071+0000","last_active":"2024-09-17T11:42:26.815097+0000","last_peered":"2024-09-17T11:42:26.815097+0000","last_clean":"2024-09-17T11:42:26.815097+0000","last_became_active":"2024-09-17T11:41:58.207692+0000","last_became_peered":"2024-09-17T11:41:58.207692+0000","last_unstale":"2024-09-17T11:42:26.815097+0000","last_undegraded":"2024-09-17T11:42:26.815097+0000","last_fullsized":"2024-09-17T11:42:26.815097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:54.089576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425245+0000","last_change":"2024-09-17T11:41:55.984519+0000","last_active":"2024-09-17T11:42:26.425245+0000","last_peered":"2024-09-17T11:42:26.425245+0000","last_clean":"2024-09-17T11:42:26.425245+0000","last_became_active":"2024-09-17T11:41:55.984191+0000","last_became_peered":"2024-09-17T11:41:55.984191+0000","last_unstale":"2024-09-17T11:42:26.425245+0000","last_undegraded":"2024-09-17T11:42:26.425245+0000","last_fullsized":"2024-09-17T11:42:26.425245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:20.663913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041493+0000","last_change":"2024-09-17T11:41:59.990171+0000","last_active":"2024-09-17T11:42:31.041493+0000","last_peered":"2024-09-17T11:42:31.041493+0000","last_clean":"2024-09-17T11:42:31.041493+0000","last_became_active":"2024-09-17T11:41:59.989863+0000","last_became_peered":"2024-09-17T11:41:59.989863+0000","last_unstale":"2024-09-17T11:42:31.041493+0000","last_undegraded":"2024-09-17T11:42:31.041493+0000","last_fullsized":"2024-09-17T11:42:31.041493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:53:50.381095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814851+0000","last_change":"2024-09-17T11:42:02.006964+0000","last_active":"2024-09-17T11:42:26.814851+0000","last_peered":"2024-09-17T11:42:26.814851+0000","last_clean":"2024-09-17T11:42:26.814851+0000","last_became_active":"2024-09-17T11:42:02.006674+0000","last_became_peered":"2024-09-17T11:42:02.006674+0000","last_unstale":"2024-09-17T11:42:26.814851+0000","last_undegraded":"2024-09-17T11:42:26.814851+0000","last_fullsized":"2024-09-17T11:42:26.814851+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:13:40.991561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"61'10","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429948+0000","last_change":"2024-09-17T11:41:58.206871+0000","last_active":"2024-09-17T11:42:26.429948+0000","last_peered":"2024-09-17T11:42:26.429948+0000","last_clean":"2024-09-17T11:42:26.429948+0000","last_became_active":"2024-09-17T11:41:58.206136+0000","last_became_peered":"2024-09-17T11:41:58.206136+0000","last_unstale":"2024-09-17T11:42:26.429948+0000","last_undegraded":"2024-09-17T11:42:26.429948+0000","last_fullsized":"2024-09-17T11:42:26.429948+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:58:00.937087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426746+0000","last_change":"2024-09-17T11:41:55.988710+0000","last_active":"2024-09-17T11:42:26.426746+0000","last_peered":"2024-09-17T11:42:26.426746+0000","last_clean":"2024-09-17T11:42:26.426746+0000","last_became_active":"2024-09-17T11:41:55.988318+0000","last_became_peered":"2024-09-17T11:41:55.988318+0000","last_unstale":"2024-09-17T11:42:26.426746+0000","last_undegraded":"2024-09-17T11:42:26.426746+0000","last_fullsized":"2024-09-17T11:42:26.426746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:16.376705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:31.041200+0000","last_change":"2024-09-17T11:41:59.995602+0000","last_active":"2024-09-17T11:42:31.041200+0000","last_peered":"2024-09-17T11:42:31.041200+0000","last_clean":"2024-09-17T11:42:31.041200+0000","last_became_active":"2024-09-17T11:41:59.995180+0000","last_became_peered":"2024-09-17T11:41:59.995180+0000","last_unstale":"2024-09-17T11:42:31.041200+0000","last_undegraded":"2024-09-17T11:42:31.041200+0000","last_fullsized":"2024-09-17T11:42:31.041200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426347+0000","last_change":"2024-09-17T11:42:02.001484+0000","last_active":"2024-09-17T11:42:26.426347+0000","last_peered":"2024-09-17T11:42:26.426347+0000","last_clean":"2024-09-17T11:42:26.426347+0000","last_became_active":"2024-09-17T11:42:02.001124+0000","last_became_peered":"2024-09-17T11:42:02.001124+0000","last_unstale":"2024-09-17T11:42:26.426347+0000","last_undegraded":"2024-09-17T11:42:26.426347+0000","last_fullsized":"2024-09-17T11:42:26.426347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:12.246081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814151+0000","last_change":"2024-09-17T11:41:57.990684+0000","last_active":"2024-09-17T11:42:26.814151+0000","last_peered":"2024-09-17T11:42:26.814151+0000","last_clean":"2024-09-17T11:42:26.814151+0000","last_became_active":"2024-09-17T11:41:57.990347+0000","last_became_peered":"2024-09-17T11:41:57.990347+0000","last_unstale":"2024-09-17T11:42:26.814151+0000","last_undegraded":"2024-09-17T11:42:26.814151+0000","last_fullsized":"2024-09-17T11:42:26.814151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:22.170034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814459+0000","last_change":"2024-09-17T11:41:55.986557+0000","last_active":"2024-09-17T11:42:26.814459+0000","last_peered":"2024-09-17T11:42:26.814459+0000","last_clean":"2024-09-17T11:42:26.814459+0000","last_became_active":"2024-09-17T11:41:55.986282+0000","last_became_peered":"2024-09-17T11:41:55.986282+0000","last_unstale":"2024-09-17T11:42:26.814459+0000","last_undegraded":"2024-09-17T11:42:26.814459+0000","last_fullsized":"2024-09-17T11:42:26.814459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:07.660835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.813800+0000","last_change":"2024-09-17T11:41:59.995587+0000","last_active":"2024-09-17T11:42:26.813800+0000","last_peered":"2024-09-17T11:42:26.813800+0000","last_clean":"2024-09-17T11:42:26.813800+0000","last_became_active":"2024-09-17T11:41:59.995304+0000","last_became_peered":"2024-09-17T11:41:59.995304+0000","last_unstale":"2024-09-17T11:42:26.813800+0000","last_undegraded":"2024-09-17T11:42:26.813800+0000","last_fullsized":"2024-09-17T11:42:26.813800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:55.392903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427445+0000","last_change":"2024-09-17T11:42:01.998771+0000","last_active":"2024-09-17T11:42:26.427445+0000","last_peered":"2024-09-17T11:42:26.427445+0000","last_clean":"2024-09-17T11:42:26.427445+0000","last_became_active":"2024-09-17T11:42:01.998466+0000","last_became_peered":"2024-09-17T11:42:01.998466+0000","last_unstale":"2024-09-17T11:42:26.427445+0000","last_undegraded":"2024-09-17T11:42:26.427445+0000","last_fullsized":"2024-09-17T11:42:26.427445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:17:23.282240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814159+0000","last_change":"2024-09-17T11:41:57.996397+0000","last_active":"2024-09-17T11:42:26.814159+0000","last_peered":"2024-09-17T11:42:26.814159+0000","last_clean":"2024-09-17T11:42:26.814159+0000","last_became_active":"2024-09-17T11:41:57.996119+0000","last_became_peered":"2024-09-17T11:41:57.996119+0000","last_unstale":"2024-09-17T11:42:26.814159+0000","last_undegraded":"2024-09-17T11:42:26.814159+0000","last_fullsized":"2024-09-17T11:42:26.814159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:18:32.721330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426769+0000","last_change":"2024-09-17T11:41:55.988135+0000","last_active":"2024-09-17T11:42:26.426769+0000","last_peered":"2024-09-17T11:42:26.426769+0000","last_clean":"2024-09-17T11:42:26.426769+0000","last_became_active":"2024-09-17T11:41:55.987846+0000","last_became_peered":"2024-09-17T11:41:55.987846+0000","last_unstale":"2024-09-17T11:42:26.426769+0000","last_undegraded":"2024-09-17T11:42:26.426769+0000","last_fullsized":"2024-09-17T11:42:26.426769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:03.881953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429376+0000","last_change":"2024-09-17T11:41:59.993716+0000","last_active":"2024-09-17T11:42:26.429376+0000","last_peered":"2024-09-17T11:42:26.429376+0000","last_clean":"2024-09-17T11:42:26.429376+0000","last_became_active":"2024-09-17T11:41:59.993373+0000","last_became_peered":"2024-09-17T11:41:59.993373+0000","last_unstale":"2024-09-17T11:42:26.429376+0000","last_undegraded":"2024-09-17T11:42:26.429376+0000","last_fullsized":"2024-09-17T11:42:26.429376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:46:29.135614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.424604+0000","last_change":"2024-09-17T11:42:01.999007+0000","last_active":"2024-09-17T11:42:26.424604+0000","last_peered":"2024-09-17T11:42:26.424604+0000","last_clean":"2024-09-17T11:42:26.424604+0000","last_became_active":"2024-09-17T11:42:01.998694+0000","last_became_peered":"2024-09-17T11:42:01.998694+0000","last_unstale":"2024-09-17T11:42:26.424604+0000","last_undegraded":"2024-09-17T11:42:26.424604+0000","last_fullsized":"2024-09-17T11:42:26.424604+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:30:09.400297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.427627+0000","last_change":"2024-09-17T11:41:58.206059+0000","last_active":"2024-09-17T11:42:26.427627+0000","last_peered":"2024-09-17T11:42:26.427627+0000","last_clean":"2024-09-17T11:42:26.427627+0000","last_became_active":"2024-09-17T11:41:58.205660+0000","last_became_peered":"2024-09-17T11:41:58.205660+0000","last_unstale":"2024-09-17T11:42:26.427627+0000","last_undegraded":"2024-09-17T11:42:26.427627+0000","last_fullsized":"2024-09-17T11:42:26.427627+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:14.209247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814689+0000","last_change":"2024-09-17T11:41:55.982646+0000","last_active":"2024-09-17T11:42:26.814689+0000","last_peered":"2024-09-17T11:42:26.814689+0000","last_clean":"2024-09-17T11:42:26.814689+0000","last_became_active":"2024-09-17T11:41:55.982369+0000","last_became_peered":"2024-09-17T11:41:55.982369+0000","last_unstale":"2024-09-17T11:42:26.814689+0000","last_undegraded":"2024-09-17T11:42:26.814689+0000","last_fullsized":"2024-09-17T11:42:26.814689+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:53:10.743086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.425854+0000","last_change":"2024-09-17T11:41:59.991592+0000","last_active":"2024-09-17T11:42:26.425854+0000","last_peered":"2024-09-17T11:42:26.425854+0000","last_clean":"2024-09-17T11:42:26.425854+0000","last_became_active":"2024-09-17T11:41:59.991318+0000","last_became_peered":"2024-09-17T11:41:59.991318+0000","last_unstale":"2024-09-17T11:42:26.425854+0000","last_undegraded":"2024-09-17T11:42:26.425854+0000","last_fullsized":"2024-09-17T11:42:26.425854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:58.569578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429549+0000","last_change":"2024-09-17T11:42:01.995676+0000","last_active":"2024-09-17T11:42:26.429549+0000","last_peered":"2024-09-17T11:42:26.429549+0000","last_clean":"2024-09-17T11:42:26.429549+0000","last_became_active":"2024-09-17T11:42:01.995369+0000","last_became_peered":"2024-09-17T11:42:01.995369+0000","last_unstale":"2024-09-17T11:42:26.429549+0000","last_undegraded":"2024-09-17T11:42:26.429549+0000","last_fullsized":"2024-09-17T11:42:26.429549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.040011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.428824+0000","last_change":"2024-09-17T11:41:57.991836+0000","last_active":"2024-09-17T11:42:26.428824+0000","last_peered":"2024-09-17T11:42:26.428824+0000","last_clean":"2024-09-17T11:42:26.428824+0000","last_became_active":"2024-09-17T11:41:57.991462+0000","last_became_peered":"2024-09-17T11:41:57.991462+0000","last_unstale":"2024-09-17T11:42:26.428824+0000","last_undegraded":"2024-09-17T11:42:26.428824+0000","last_fullsized":"2024-09-17T11:42:26.428824+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:04:31.343026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426147+0000","last_change":"2024-09-17T11:41:55.988729+0000","last_active":"2024-09-17T11:42:26.426147+0000","last_peered":"2024-09-17T11:42:26.426147+0000","last_clean":"2024-09-17T11:42:26.426147+0000","last_became_active":"2024-09-17T11:41:55.987114+0000","last_became_peered":"2024-09-17T11:41:55.987114+0000","last_unstale":"2024-09-17T11:42:26.426147+0000","last_undegraded":"2024-09-17T11:42:26.426147+0000","last_fullsized":"2024-09-17T11:42:26.426147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:11.715778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.815185+0000","last_change":"2024-09-17T11:41:59.991583+0000","last_active":"2024-09-17T11:42:26.815185+0000","last_peered":"2024-09-17T11:42:26.815185+0000","last_clean":"2024-09-17T11:42:26.815185+0000","last_became_active":"2024-09-17T11:41:59.991304+0000","last_became_peered":"2024-09-17T11:41:59.991304+0000","last_unstale":"2024-09-17T11:42:26.815185+0000","last_undegraded":"2024-09-17T11:42:26.815185+0000","last_fullsized":"2024-09-17T11:42:26.815185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:21:13.985381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426228+0000","last_change":"2024-09-17T11:42:02.004610+0000","last_active":"2024-09-17T11:42:26.426228+0000","last_peered":"2024-09-17T11:42:26.426228+0000","last_clean":"2024-09-17T11:42:26.426228+0000","last_became_active":"2024-09-17T11:42:02.003548+0000","last_became_peered":"2024-09-17T11:42:02.003548+0000","last_unstale":"2024-09-17T11:42:26.426228+0000","last_undegraded":"2024-09-17T11:42:26.426228+0000","last_fullsized":"2024-09-17T11:42:26.426228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:25:57.742990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.814243+0000","last_change":"2024-09-17T11:42:02.003309+0000","last_active":"2024-09-17T11:42:26.814243+0000","last_peered":"2024-09-17T11:42:26.814243+0000","last_clean":"2024-09-17T11:42:26.814243+0000","last_became_active":"2024-09-17T11:42:02.002973+0000","last_became_peered":"2024-09-17T11:42:02.002973+0000","last_unstale":"2024-09-17T11:42:26.814243+0000","last_undegraded":"2024-09-17T11:42:26.814243+0000","last_fullsized":"2024-09-17T11:42:26.814243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:42:00.977682+0000","last_clean_scrub_stamp":"2024-09-17T11:42:00.977682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:04.069209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.429140+0000","last_change":"2024-09-17T11:41:55.981713+0000","last_active":"2024-09-17T11:42:26.429140+0000","last_peered":"2024-09-17T11:42:26.429140+0000","last_clean":"2024-09-17T11:42:26.429140+0000","last_became_active":"2024-09-17T11:41:55.981134+0000","last_became_peered":"2024-09-17T11:41:55.981134+0000","last_unstale":"2024-09-17T11:42:26.429140+0000","last_undegraded":"2024-09-17T11:42:26.429140+0000","last_fullsized":"2024-09-17T11:42:26.429140+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:54.967702+0000","last_clean_scrub_stamp":"2024-09-17T11:41:54.967702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:30:55.599888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'11","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.816171+0000","last_change":"2024-09-17T11:41:57.986999+0000","last_active":"2024-09-17T11:42:26.816171+0000","last_peered":"2024-09-17T11:42:26.816171+0000","last_clean":"2024-09-17T11:42:26.816171+0000","last_became_active":"2024-09-17T11:41:57.986681+0000","last_became_peered":"2024-09-17T11:41:57.986681+0000","last_unstale":"2024-09-17T11:42:26.816171+0000","last_undegraded":"2024-09-17T11:42:26.816171+0000","last_fullsized":"2024-09-17T11:42:26.816171+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:56.970815+0000","last_clean_scrub_stamp":"2024-09-17T11:41:56.970815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:03.860041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T11:42:26.426610+0000","last_change":"2024-09-17T11:41:59.990661+0000","last_active":"2024-09-17T11:42:26.426610+0000","last_peered":"2024-09-17T11:42:26.426610+0000","last_clean":"2024-09-17T11:42:26.426610+0000","last_became_active":"2024-09-17T11:41:59.990341+0000","last_became_peered":"2024-09-17T11:41:59.990341+0000","last_unstale":"2024-09-17T11:42:26.426610+0000","last_undegraded":"2024-09-17T11:42:26.426610+0000","last_fullsized":"2024-09-17T11:42:26.426610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T11:41:58.974119+0000","last_clean_scrub_stamp":"2024-09-17T11:41:58.974119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:21:49.240224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":31,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397515,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1232896,"data_stored":830670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":830578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789974,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713356,"statfs":{"total":95995035648,"available":95962476544,"internally_reserved":0,"allocated":622592,"data_stored":240556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986204,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31820,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713332,"statfs":{"total":95995035648,"available":95962451968,"internally_reserved":0,"allocated":647168,"data_stored":240245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149728,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717484,"statfs":{"total":95995035648,"available":95966703616,"internally_reserved":0,"allocated":655360,"data_stored":241088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411365,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31792,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713360,"statfs":{"total":95995035648,"available":95962480640,"internally_reserved":0,"allocated":618496,"data_stored":239801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717456,"statfs":{"total":95995035648,"available":95966674944,"internally_reserved":0,"allocated":684032,"data_stored":241425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1232896,"data_stored":830366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T11:42:43.738 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T11:42:43.739 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T11:42:43.739 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T11:42:43.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169 shell --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T11:42:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:43 smithi049 ceph-mon[40223]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:43 smithi049 ceph-mon[40223]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:43.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:43 smithi049 ceph-mon[30017]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:43.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:43 smithi049 ceph-mon[30017]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:43 smithi129 ceph-mon[33364]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T11:42:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:43 smithi129 ceph-mon[33364]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-17T11:42:44.521 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 -- 172.21.15.49:0/3620328032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c109130 msgr2=0x7fc38c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 --2- 172.21.15.49:0/3620328032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c109130 0x7fc38c10f930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc384009be0 tx=0x7fc38402a6c0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 -- 172.21.15.49:0/3620328032 shutdown_connections 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 --2- 172.21.15.49:0/3620328032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c109130 0x7fc38c10f930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 --2- 172.21.15.49:0/3620328032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 0x7fc38c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 --2- 172.21.15.49:0/3620328032 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c102da0 0x7fc38c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 -- 172.21.15.49:0/3620328032 >> 172.21.15.49:0/3620328032 conn(0x7fc38c0fe810 msgr2=0x7fc38c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:45.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 -- 172.21.15.49:0/3620328032 shutdown_connections 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.111+0000 7fc39146e640 1 -- 172.21.15.49:0/3620328032 wait complete. 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 Processor -- start 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 -- start start 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 0x7fc38c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c109130 0x7fc38c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c1124c0 con 0x7fc38c102da0 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c112340 con 0x7fc38c1087b0 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc39146e640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fc38c112640 con 0x7fc38c109130 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42214/0 (socket says 172.21.15.49:42214) 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38b7fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c109130 0x7fc38c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38affd640 1 -- 172.21.15.49:0/1310393604 learned_addr learned my addr 172.21.15.49:0/1310393604 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38a7fc640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 0x7fc38c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38affd640 1 -- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c109130 msgr2=0x7fc38c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.112+0000 7fc38affd640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c109130 0x7fc38c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc38affd640 1 -- 172.21.15.49:0/1310393604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 msgr2=0x7fc38c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc38affd640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 0x7fc38c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc38affd640 1 -- 172.21.15.49:0/1310393604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c19e980 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc38affd640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9c0 tx=0x7fc37c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00cd80 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fc37c0035f0 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0044a0 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c19ec70 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.113+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c1a5330 con 0x7fc38c102da0 2024-09-17T11:42:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.114+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c103180 con 0x7fc38c102da0 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.115+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fc37c004000 con 0x7fc38c102da0 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.115+0000 7fc373fff640 1 --2- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 0x7fc37407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.116+0000 7fc38a7fc640 1 --2- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 0x7fc37407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.116+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6172+0+0 (secure 0 0 0) 0x7fc37c09e1a0 con 0x7fc38c102da0 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.116+0000 7fc38a7fc640 1 --2- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 0x7fc37407d290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc38c102070 tx=0x7fc38000a2d0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:42:45.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.120+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc37c069930 con 0x7fc38c102da0 2024-09-17T11:42:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.346+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc38c1a55f0 con 0x7fc38c102da0 2024-09-17T11:42:45.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.347+0000 7fc373fff640 1 -- 172.21.15.49:0/1310393604 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc37c06e7e0 con 0x7fc38c102da0 2024-09-17T11:42:45.348 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-09-17T11:42:45.348 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T11:42:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 msgr2=0x7fc37407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 0x7fc37407d290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc38c102070 tx=0x7fc38000a2d0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 msgr2=0x7fc38c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9c0 tx=0x7fc37c00edc0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 shutdown_connections 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc37407add0 0x7fc37407d290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc38c109130 0x7fc38c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc38c1087b0 0x7fc38c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 --2- 172.21.15.49:0/1310393604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc38c102da0 0x7fc38c1988d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 >> 172.21.15.49:0/1310393604 conn(0x7fc38c0fe810 msgr2=0x7fc38c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 shutdown_connections 2024-09-17T11:42:45.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-09-17T11:42:45.349+0000 7fc39146e640 1 -- 172.21.15.49:0/1310393604 wait complete. 2024-09-17T11:42:45.874 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:45 smithi049 ceph-mon[40223]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:45.874 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1310393604' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-17T11:42:45.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:45 smithi049 ceph-mon[30017]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:45.875 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1310393604' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-17T11:42:46.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:45 smithi129 ceph-mon[33364]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-17T11:42:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1310393604' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-17T11:42:46.639 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T11:42:46.639 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T11:42:46.639 INFO:teuthology.run_tasks:Running task workunit... 2024-09-17T11:42:46.654 INFO:tasks.workunit:Pulling workunits from ref 2b698643d05f73e79c2d2888228947a84a4ac0bd 2024-09-17T11:42:46.655 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-17T11:42:46.655 DEBUG:teuthology.orchestra.run.smithi049:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-17T11:42:46.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T11:42:46.673 INFO:teuthology.orchestra.run.smithi049.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T11:42:46.673 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-17T11:42:46.729 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-17T11:42:46.730 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-17T11:42:46.786 INFO:tasks.workunit:timeout=3h 2024-09-17T11:42:46.786 INFO:tasks.workunit:cleanup=True 2024-09-17T11:42:46.786 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2b698643d05f73e79c2d2888228947a84a4ac0bd 2024-09-17T11:42:46.842 INFO:tasks.workunit.client.0.smithi049.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-17T11:42:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:42:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:42:47.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:42:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:47 smithi049 ceph-mon[40223]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:47.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:47 smithi049 ceph-mon[30017]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:47 smithi129 ceph-mon[33364]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:48.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:48.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:48 smithi049 systemd[1]: Starting Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:42:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:49.243 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:48 smithi049 podman[73889]: 2024-09-17 11:42:48.979241128 +0000 UTC m=+0.093047879 volume create e397af2d4bb75139ae8a75443e3dfb9322f52edbbd22f51750912c71630ab141 2024-09-17T11:42:49.243 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:48 smithi049 podman[73889]: 2024-09-17 11:42:48.902480142 +0000 UTC m=+0.016286904 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-17T11:42:49.243 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 podman[73889]: 2024-09-17 11:42:49.241747603 +0000 UTC m=+0.355554358 container create f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[40223]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[30017]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 podman[73889]: 2024-09-17 11:42:49.550779027 +0000 UTC m=+0.664585786 container init f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 podman[73889]: 2024-09-17 11:42:49.555966765 +0000 UTC m=+0.669773515 container start f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.567Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.567Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.568Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.49 port=9094 2024-09-17T11:42:49.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.569Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.591Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.592Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.593Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:49.593Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 bash[73889]: f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c 2024-09-17T11:42:49.902 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:49 smithi049 systemd[1]: Started Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:42:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:49 smithi129 ceph-mon[33364]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:49 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:49 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:49 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:49 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:50 smithi049 ceph-mon[40223]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T11:42:51.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:50 smithi049 ceph-mon[30017]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T11:42:51.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:50 smithi129 ceph-mon[33364]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T11:42:51.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:51] ENGINE Bus STOPPING 2024-09-17T11:42:51.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:51.569Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000138663s 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[40223]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[30017]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:51.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:51.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-17T11:42:51.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:51 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:51 smithi129 ceph-mon[33364]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:51 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:51 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:51 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-17T11:42:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:51 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:52.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:52] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T11:42:52.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:52] ENGINE Bus STOPPED 2024-09-17T11:42:52.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:52] ENGINE Bus STARTING 2024-09-17T11:42:52.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:52] ENGINE Serving on http://:::9283 2024-09-17T11:42:52.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:42:52] ENGINE Bus STARTED 2024-09-17T11:42:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T11:42:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[40223]: Deploying daemon grafana.a on smithi129 2024-09-17T11:42:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T11:42:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[30017]: Deploying daemon grafana.a on smithi129 2024-09-17T11:42:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:52 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T11:42:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:52 smithi129 ceph-mon[33364]: Deploying daemon grafana.a on smithi129 2024-09-17T11:42:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:52 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:42:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:52 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:42:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:53 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:53 smithi049 ceph-mon[40223]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:53 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:53 smithi049 ceph-mon[30017]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:53 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:42:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:53 smithi129 ceph-mon[33364]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:56.148 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:55 smithi049 ceph-mon[40223]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:56.148 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:55 smithi049 ceph-mon[30017]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:55 smithi129 ceph-mon[33364]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:42:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:42:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:42:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:42:57.897 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:42:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:42:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:57 smithi049 ceph-mon[40223]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:57 smithi049 ceph-mon[30017]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:57 smithi129 ceph-mon[33364]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:42:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:42:59.084 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:59.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:42:59.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:42:59 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:42:59.571Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002314379s 2024-09-17T11:43:00.150 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:42:59 smithi129 ceph-mon[33364]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:00.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:42:59 smithi049 ceph-mon[30017]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:42:59 smithi049 ceph-mon[40223]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:01.677 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:01 smithi129 systemd[1]: Starting Ceph grafana.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:43:02.075 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:01 smithi129 ceph-mon[33364]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:02.075 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:01 smithi129 podman[62819]: 2024-09-17 11:43:01.730891883 +0000 UTC m=+0.101183005 container create 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T11:43:02.075 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:01 smithi129 podman[62819]: 2024-09-17 11:43:01.646116023 +0000 UTC m=+0.016407147 image pull c8b91775d855b99270fc5d22f3c6737e8cca01ef4c25c8b0362295e0746fa39b quay.io/ceph/grafana:10.4.0 2024-09-17T11:43:02.076 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:01 smithi129 podman[62819]: 2024-09-17 11:43:01.958137416 +0000 UTC m=+0.328428531 container init 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T11:43:02.076 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:01 smithi129 podman[62819]: 2024-09-17 11:43:01.962898759 +0000 UTC m=+0.333189875 container start 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T11:43:02.326 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 bash[62819]: 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 systemd[1]: Started Ceph grafana.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085573669Z level=info msg="Starting Grafana" version=10.4.0 commit=03f502a94d17f7dc4e6c34acdf8428aedd986e4c branch=HEAD compiled=2024-09-17T11:43:02Z 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085800323Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085808363Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085811717Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085815024Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085817902Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085820928Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-17T11:43:02.327 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085824359Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085827699Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085830713Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085833676Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085836567Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085840027Z level=info msg=Target target=[all] 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085846956Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085849895Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-17T11:43:02.328 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085852841Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085856028Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085858834Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=settings t=2024-09-17T11:43:02.085861563Z level=info msg="App mode production" 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore t=2024-09-17T11:43:02.086127529Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore t=2024-09-17T11:43:02.08614069Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.086616612Z level=info msg="Starting DB migrations" 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.087462242Z level=info msg="Executing migration" id="create migration_log table" 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.088315483Z level=info msg="Migration successfully executed" id="create migration_log table" duration=852.542µs 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.088758882Z level=info msg="Executing migration" id="create user table" 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.089325749Z level=info msg="Migration successfully executed" id="create user table" duration=566.73µs 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.089744654Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-17T11:43:02.329 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.090249241Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=504.78µs 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.090633669Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.091131415Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=498.844µs 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.091564202Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.092071984Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=507.999µs 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.092567267Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.093038657Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=471.812µs 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.093381242Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.094932959Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.552803ms 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.095305342Z level=info msg="Executing migration" id="create user table v2" 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.095818968Z level=info msg="Migration successfully executed" id="create user table v2" duration=513.428µs 2024-09-17T11:43:02.330 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.096172912Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.09664253Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=469.485µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.097038143Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.09748581Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=449.215µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.097945607Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.098246487Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=295.922µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.098597027Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.098987114Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=389.912µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.099317503Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.100041863Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=723.742µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.100391013Z level=info msg="Executing migration" id="Update user table charset" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.100407101Z level=info msg="Migration successfully executed" id="Update user table charset" duration=16.746µs 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.100790814Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-17T11:43:02.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.101575137Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=783.828µs 2024-09-17T11:43:02.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.101930353Z level=info msg="Executing migration" id="Add missing user data" 2024-09-17T11:43:02.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.102075787Z level=info msg="Migration successfully executed" id="Add missing user data" duration=145.551µs 2024-09-17T11:43:02.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.102479769Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-17T11:43:02.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.103211814Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=731.486µs 2024-09-17T11:43:02.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.103580407Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-17T11:43:02.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.104050487Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=469.824µs 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.104393026Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.105122474Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=728.844µs 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.105469488Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.11076841Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.296223ms 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.11127972Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.112037603Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=755.966µs 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.112390574Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.112524672Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=134.258µs 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.112909765Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.113370761Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=460.932µs 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.113801851Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-17T11:43:02.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.114380935Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=578.712µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.114804657Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.115294165Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=489.346µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.115671647Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.116167645Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=479.6µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.1165668Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.117044114Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=477.24µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.117418668Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.117944772Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=526.004µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.118308771Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.118326222Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=16.595µs 2024-09-17T11:43:02.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.118770963Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.119262043Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=490.807µs 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.119655131Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.120133954Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=478.731µs 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.120490984Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.120983582Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=491.128µs 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.121341781Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.121794242Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=452.158µs 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.122166498Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.124385975Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=2.21718ms 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.124820878Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.125401547Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=578.127µs 2024-09-17T11:43:02.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.125789007Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.126301761Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=512.621µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.126708374Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.127216013Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=507.912µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.127568805Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.128040567Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=471.452µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.130458836Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.130986317Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=527.521µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.131399518Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.131746109Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=346.426µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.132153865Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.132658156Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=503.888µs 2024-09-17T11:43:02.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.133066762Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.133387519Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=321.162µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.1338447Z level=info msg="Executing migration" id="create star table" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.134442159Z level=info msg="Migration successfully executed" id="create star table" duration=597.581µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.135757576Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.136257271Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=499.778µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.138527361Z level=info msg="Executing migration" id="create org table v1" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.139076391Z level=info msg="Migration successfully executed" id="create org table v1" duration=548.167µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.140979352Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.14161004Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=632.131µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.143795234Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.14428566Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=490.953µs 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.146499144Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-17T11:43:02.339 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.147022773Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=523.557µs 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.149157812Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.149859087Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=700.836µs 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.151834484Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.15236972Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=535.415µs 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.154463788Z level=info msg="Executing migration" id="Update org table charset" 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.154486963Z level=info msg="Migration successfully executed" id="Update org table charset" duration=23.779µs 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.156994685Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.15701539Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=21.172µs 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.159655466Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-17T11:43:02.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.159776227Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=120.165µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.162308636Z level=info msg="Executing migration" id="create dashboard table" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.162818287Z level=info msg="Migration successfully executed" id="create dashboard table" duration=509.414µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.164984302Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.165610758Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=628.896µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.167499822Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.168042457Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=542.266µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.170124435Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.170565878Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=441.401µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.172804013Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.173330691Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=527.51µs 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.175552987Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-17T11:43:02.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.176040345Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=485.976µs 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.178049154Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.181466922Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.417517ms 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.181851311Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.182365593Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=514.259µs 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.182734482Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.183222663Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=488.03µs 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.183610197Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.184108002Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=497.407µs 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.184470566Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.184721468Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=249.845µs 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.185329013Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-17T11:43:02.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.185924744Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=595.674µs 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.18628064Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.18633435Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=47.666µs 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.187136272Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.188255902Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.119207ms 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.188639984Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.18969969Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.058368ms 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.190107216Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.191210039Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.101281ms 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.191598202Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.192095806Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=497.438µs 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.192482858Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.193569809Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.085676ms 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.193960356Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-17T11:43:02.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.194470021Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=508.352µs 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.194841455Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.19533688Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=495.11µs 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.195749814Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.19577505Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=24.973µs 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.19675895Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.19677526Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=17.093µs 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.197466215Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.198670163Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.203671ms 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.199070755Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.200278767Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.206646ms 2024-09-17T11:43:02.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.200710902Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.201899572Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.188414ms 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.202273307Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.203467488Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.195027ms 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.203846472Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.204022623Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=176.12µs 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.204748728Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.205287075Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=537.028µs 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.205711518Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.206220368Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=508.979µs 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.206577967Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-17T11:43:02.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.206598509Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=20.619µs 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.207572037Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.208168418Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=597.371µs 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.208715714Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.209290944Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=576.071µs 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.20973207Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.212913225Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.182077ms 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.213272925Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.213725135Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=451.055µs 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.214132623Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.214610651Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=477.998µs 2024-09-17T11:43:02.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.214969624Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.215450221Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=480.276µs 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.215804518Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.216018455Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=213.972µs 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.21668727Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.217050707Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=363.321µs 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.217549222Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.218725397Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.175896ms 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.219088038Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.219548066Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=459.915µs 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.219932559Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.220039839Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=107.321µs 2024-09-17T11:43:02.347 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.220830473Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.220943958Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=113.551µs 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.221796475Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.222302255Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=491.923µs 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.224644584Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.225910944Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.266025ms 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.227368872Z level=info msg="Executing migration" id="create data_source table" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.228019585Z level=info msg="Migration successfully executed" id="create data_source table" duration=649.219µs 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.230076159Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.230565964Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=491.116µs 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.233019879Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.23362094Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=602.953µs 2024-09-17T11:43:02.348 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.235798424Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.2363509Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=554.809µs 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.238508673Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.239002484Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=494.204µs 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.241358143Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.244907643Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.551715ms 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.245352016Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.245960492Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=608.274µs 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.248143586Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.248661981Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=518.22µs 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.25081187Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.251369209Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=557.255µs 2024-09-17T11:43:02.349 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.25365572Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.254087998Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=432.156µs 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.256144132Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.257507629Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.362228ms 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.259231315Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.260625409Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.394928ms 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.262264369Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.262337758Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=76.413µs 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.264969855Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.265102253Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=132.592µs 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.267591802Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-17T11:43:02.350 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.268985412Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.393278ms 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.270410396Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.270593786Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=186.409µs 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.273168456Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.27333102Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=163.996µs 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.275739495Z level=info msg="Executing migration" id="Add uid column" 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.277154813Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.414931ms 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.27840655Z level=info msg="Executing migration" id="Update uid value" 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.278548333Z level=info msg="Migration successfully executed" id="Update uid value" duration=141.88µs 2024-09-17T11:43:02.351 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.281010059Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.281529345Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=519.224µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.283770763Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.284288132Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=517.225µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.286412427Z level=info msg="Executing migration" id="create api_key table" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.286951082Z level=info msg="Migration successfully executed" id="create api_key table" duration=538.369µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.288953388Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.289500649Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=547.447µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.291469988Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.291991676Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=521.549µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.294190442Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.29479322Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=604.752µs 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.296795144Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-17T11:43:02.352 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.297307054Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=511.701µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.2995671Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.300146413Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=581.563µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.30052598Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.30102427Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=498.125µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.303526482Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.307362664Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.83739ms 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.30772657Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.308240199Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=513.852µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.310624875Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.31120795Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=585.308µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.311567707Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.312070178Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=502.492µs 2024-09-17T11:43:02.353 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.314301155Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.314836941Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=537.712µs 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.317046435Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.317292717Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=245.996µs 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.317814989Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.318373566Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=560.829µs 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.318760037Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.318778628Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=18.699µs 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.319540638Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.321034112Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.493118ms 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.321390164Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.322858177Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.467854ms 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.323221185Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-17T11:43:02.354 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.323332685Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=111.713µs 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.323720362Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.325257793Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.537243ms 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.325649857Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.327220619Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.570505ms 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.327574389Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.328110798Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=536.259µs 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.328460775Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.328852868Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=393.113µs 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.329212433Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.32978893Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=576.335µs 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.330154563Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.330709862Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=555.111µs 2024-09-17T11:43:02.355 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.331081271Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-17T11:43:02.356 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.331647483Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=566.406µs 2024-09-17T11:43:02.356 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.332079283Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-17T11:43:02.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:01 smithi049 ceph-mon[30017]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:01 smithi049 ceph-mon[40223]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:02.576 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.332616042Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=536.553µs 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.332967904Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.33301733Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=49.116µs 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.333728214Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.333746509Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=18.47µs 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.334512242Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.336183579Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.670914ms 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.336542158Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.338226275Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.683909ms 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.338606493Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.338655683Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=50.214µs 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.339449032Z level=info msg="Executing migration" id="create quota table v1" 2024-09-17T11:43:02.577 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.339940372Z level=info msg="Migration successfully executed" id="create quota table v1" duration=490.993µs 2024-09-17T11:43:02.578 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.340293529Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-17T11:43:02.578 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.340801038Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=507.203µs 2024-09-17T11:43:02.578 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.341154301Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-17T11:43:02.578 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.34117266Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=18.558µs 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.341572195Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.342078898Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=506.336µs 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.342441224Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.342989679Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=548.009µs 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.343334567Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-17T11:43:02.579 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.345026323Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.691329ms 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.345368805Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.345400786Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=31.919µs 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.345818208Z level=info msg="Executing migration" id="create session table" 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.346398922Z level=info msg="Migration successfully executed" id="create session table" duration=579.276µs 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.346772258Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.346834825Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=62.861µs 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.347589831Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.347666557Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=76.73µs 2024-09-17T11:43:02.580 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.348439324Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.348940391Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=501.297µs 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.349308785Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.349797754Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=489.254µs 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.350167009Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.350203959Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=38.085µs 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.35105919Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.3510743Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=15.866µs 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.352035751Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-17T11:43:02.581 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.353822239Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.788682ms 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.354220771Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.356046902Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.825826ms 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.356465386Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.356538904Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=71.693µs 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.359319272Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.359402841Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=83.165µs 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.362105286Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.362729728Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=626.987µs 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.363118248Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.363133841Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=16.458µs 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.363552375Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-17T11:43:02.582 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.365407033Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.856302ms 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.36578645Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.36591058Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=124.28µs 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.3663183Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.36818184Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.863362ms 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.368566961Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.370412797Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.845643ms 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.370795426Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.370841111Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=45.368µs 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.371328449Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.371992303Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=664.511µs 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.372380556Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.372946896Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=564.46µs 2024-09-17T11:43:02.583 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.373328339Z level=info msg="Executing migration" id="create alert table v1" 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.37406049Z level=info msg="Migration successfully executed" id="create alert table v1" duration=731.963µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.374440599Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.375062842Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=623.455µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.377137486Z level=info msg="Executing migration" id="add index alert state" 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.377674673Z level=info msg="Migration successfully executed" id="add index alert state" duration=537.128µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.378036449Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.378576985Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=540.447µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.378960162Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.379413415Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=454.773µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.379786642Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.380372378Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=585.664µs 2024-09-17T11:43:02.584 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.380748573Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.381288062Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=539.431µs 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.381630495Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.38728215Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.6525ms 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.387684541Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.388162334Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=477.848µs 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.388509214Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.389056951Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=547.518µs 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.389391802Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.38958406Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=192.191µs 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.389904068Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.390271051Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=366.94µs 2024-09-17T11:43:02.585 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.390617474Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.391093306Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=475.592µs 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.391420222Z level=info msg="Executing migration" id="Add column is_default" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.393463302Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.042912ms 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.393815778Z level=info msg="Executing migration" id="Add column frequency" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.395996045Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.178408ms 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.396349078Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.398649337Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.299901ms 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.399003723Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.401061868Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.057826ms 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.401397081Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.401927241Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=529.818µs 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.402283084Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.402302731Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=20.563µs 2024-09-17T11:43:02.586 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.402711855Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.402731098Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=20.288µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.40315045Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.403625074Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=474.622µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.404021827Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.404567857Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=545.869µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.404969535Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.405542306Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=572.838µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.407519072Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.408104461Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=585.235µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.408500482Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.40905852Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=557.798µs 2024-09-17T11:43:02.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.410988315Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.413204049Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.214306ms 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.413576549Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.415768604Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.191857ms 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.416154663Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.416290281Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=135.63µs 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.416691334Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.417236264Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=545.76µs 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.419450829Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.420036157Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=585.04µs 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.422043767Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.424344794Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.303122ms 2024-09-17T11:43:02.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.424740117Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.424785728Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=45.304µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.42526506Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.425789974Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=524.526µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.428165707Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.428855613Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=691.984µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.43039823Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.430477579Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=78.706µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.431354258Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.432114065Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=761.126µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.432574464Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.433123251Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=549.621µs 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.433541266Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T11:43:02.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.434089267Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=547.777µs 2024-09-17T11:43:02.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.434505943Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.435056543Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=551.856µs 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.435477712Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.436084594Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=606.497µs 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.436509745Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.437129902Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=619.856µs 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.437553835Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.43757864Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=24.844µs 2024-09-17T11:43:02.591 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.438336067Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.440805571Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.469241ms 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.441195759Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.441757697Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=561.87µs 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.442124825Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.444460662Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.335508ms 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.444845601Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.44535243Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=506.642µs 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.445730235Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.446324652Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=593.615µs 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.44669467Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.447256643Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=561.963µs 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.447621045Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T11:43:02.592 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.454034242Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.412844ms 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.454416505Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.454929179Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=511.095µs 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.455291975Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.455861652Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=569.508µs 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.456263402Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.456474767Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=213.052µs 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.45685243Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.457250125Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=397.57µs 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.457663395Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.457790041Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=126.9µs 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.458548853Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.460884302Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.33506ms 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.461272036Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T11:43:02.593 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.463612691Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.340661ms 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.464008655Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.464568929Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=560.277µs 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.464944076Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.465483609Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=539.384µs 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.465850072Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.466017894Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=167.786µs 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.466686415Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.469045489Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.357112ms 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.469424939Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.469988397Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=564.889µs 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.470393443Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T11:43:02.594 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.47051431Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=120.253µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.471336821Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.471692561Z level=info msg="Migration successfully executed" id="Move region to single row" duration=358.169µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.472110144Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.472656511Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=544.629µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.473101526Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.473659415Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=557.912µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.474051798Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.474626185Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=574.168µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.475001187Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.475567545Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=566.133µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.475946046Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.476483553Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=537.398µs 2024-09-17T11:43:02.595 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.476840737Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.477394488Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=551.932µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.47775092Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.47779708Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=45.903µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.478563214Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.479136051Z level=info msg="Migration successfully executed" id="create test_data table" duration=572.743µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.479528363Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.48007829Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=549.83µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.48047685Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.48102989Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=552.924µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.483308618Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.483909889Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=599.157µs 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.486210504Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T11:43:02.596 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.486359338Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=148.706µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.488832618Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.489101824Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=268.822µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.489453865Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.489493695Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=41.148µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.492217913Z level=info msg="Executing migration" id="create team table" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.49272436Z level=info msg="Migration successfully executed" id="create team table" duration=506.18µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.49310888Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.493741425Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=632.588µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.495072779Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.495639266Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=566.226µs 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.497778919Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.50035295Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.576365ms 2024-09-17T11:43:02.597 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.500761248Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.50090099Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=139.293µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.503655697Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.504265078Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=608.995µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.506324002Z level=info msg="Executing migration" id="create team member table" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.506861476Z level=info msg="Migration successfully executed" id="create team member table" duration=537.622µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.509124146Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.509718702Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=596.981µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.510117919Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.510672037Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=552.641µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.513113116Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.513734807Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=623.67µs 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.515863143Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T11:43:02.598 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.518680279Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.821555ms 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.519059967Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.521829458Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.769296ms 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.522209768Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.524829342Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.619479ms 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.525215323Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.525836302Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=620.869µs 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.527717735Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.52833474Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=616.806µs 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.528693397Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.529337518Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=643.736µs 2024-09-17T11:43:02.599 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.531552664Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.532169088Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=616.087µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.534228865Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.53479175Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=562.715µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.536777548Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.537382656Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=604.915µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.539649724Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.540302807Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=653.079µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.542123173Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.542742839Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=619.492µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.544557858Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.544899642Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=342.017µs 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.547099998Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T11:43:02.600 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.547245308Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=145.729µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.549768195Z level=info msg="Executing migration" id="create tag table" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.550292069Z level=info msg="Migration successfully executed" id="create tag table" duration=524.215µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.552465319Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.553039562Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=574.583µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.555151138Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.555625028Z level=info msg="Migration successfully executed" id="create login attempt table" duration=474.707µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.557625439Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.55821835Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=592.635µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.560324896Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.561007212Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=686.347µs 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.56296662Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.570955981Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=7.988977ms 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.571352296Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T11:43:02.601 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.57184597Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=493.095µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.573718364Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.574320811Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=602.824µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.574692184Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.574917Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=225.074µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.57553253Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.576101366Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=568.3µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.576540708Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.577215016Z level=info msg="Migration successfully executed" id="create user auth table" duration=674.209µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.57765463Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.578491278Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=836.305µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.57893778Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.578990652Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=53.447µs 2024-09-17T11:43:02.602 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.579713332Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T11:43:02.827 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.582822277Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.108969ms 2024-09-17T11:43:02.827 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.583209444Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T11:43:02.827 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.58617078Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.960908ms 2024-09-17T11:43:02.827 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.586540585Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T11:43:02.827 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.589452999Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.912152ms 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.589814798Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.592736522Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.92137ms 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.593103579Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.593693806Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=590.217µs 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.594093152Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.597090558Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.989537ms 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.597458168Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.59801657Z level=info msg="Migration successfully executed" id="create server_lock table" duration=556.499µs 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.598401364Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.599007392Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=605.99µs 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.599388608Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T11:43:02.828 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.599977763Z level=info msg="Migration successfully executed" id="create user auth token table" duration=588.966µs 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.600339042Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.600948303Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=607.237µs 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.601306951Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.601916113Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=608.979µs 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.602277405Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.602940181Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=662.73µs 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.603318612Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.606499795Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.180758ms 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.606852299Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.60746706Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=614.722µs 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.607858178Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T11:43:02.829 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.608447052Z level=info msg="Migration successfully executed" id="create cache_data table" duration=588.738µs 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.60893209Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.609555322Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=623.115µs 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.609953506Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.61054758Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=593.978µs 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.612731869Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T11:43:02.830 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.613390912Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=658.622µs 2024-09-17T11:43:02.831 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.615441094Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T11:43:02.831 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.615496451Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=55.841µs 2024-09-17T11:43:02.831 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.618144682Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.618207039Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=62.053µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.621090814Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.621751301Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=662.968µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.623900131Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.624527805Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=627.61µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.626885249Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.627531045Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=644.583µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.629614104Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.629714472Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=101.468µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.632410656Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.63314999Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=725.441µs 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.63527991Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T11:43:02.832 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.635892507Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=614.835µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.638075982Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.63872111Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=647.054µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.640629575Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.641242071Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=613.81µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.64362808Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.646941754Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.315655ms 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.647268619Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.647907154Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=638.183µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.650065333Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.650217627Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=153.227µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.652795846Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.653418549Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=625.113µs 2024-09-17T11:43:02.833 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.655458231Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.656133275Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=681.528µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.658253307Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.658921212Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=670.157µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.661033122Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.6611362Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=104.141µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.663729449Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.664456602Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=729.235µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.666183396Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.66677454Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=590.806µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.66900602Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.66968469Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=680.759µs 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.671714957Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T11:43:02.834 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.672484571Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=770.037µs 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.674083234Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.677461985Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.378454ms 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.677806567Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.67838873Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=582.083µs 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.680553521Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.681155842Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=602.189µs 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.68312621Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.700537202Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=17.410735ms 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.700927395Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.717090741Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=16.163047ms 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.717437971Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T11:43:02.835 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.718035787Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=597.618µs 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.720194074Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.720889921Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=701.773µs 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.72268694Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.72582684Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.139339ms 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.72618668Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.729408909Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.221796ms 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.729776772Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.73039231Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=615.595µs 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.732610419Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.733251006Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=640.287µs 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.735020846Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.735629083Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=608.072µs 2024-09-17T11:43:02.836 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.737916596Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.738637889Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=723.411µs 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.739000544Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.73903805Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=37.836µs 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.73986477Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.743307199Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.442074ms 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.743667533Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.746992117Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.324085ms 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.747338553Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.750642927Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.302608ms 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.751005687Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.751565983Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=559.989µs 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.751906577Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T11:43:02.837 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.752521912Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=614.956µs 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.752873567Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.756230313Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.361681ms 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.756598677Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.759898649Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.299699ms 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.760263815Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.760904565Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=640.426µs 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.76124778Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.764529626Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.281539ms 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.764904772Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.76826304Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.358034ms 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.768630781Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.76866845Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=37.896µs 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.769500633Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T11:43:02.838 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.770227098Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=726.522µs 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.770594079Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.771269402Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=676.223µs 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.771624522Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.772314728Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=689.856µs 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.772721053Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.772839857Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=119.44µs 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.773512335Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.777086152Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.571637ms 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.777433155Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.780966942Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.533345ms 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.781314745Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.784783452Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.467019ms 2024-09-17T11:43:02.839 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.785176051Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.788701821Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.507612ms 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.789079376Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.792503291Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.423657ms 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.792864344Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.792921473Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=57.54µs 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.793902916Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.794448273Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=549.554µs 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.794811436Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.79830225Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.490633ms 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.798673106Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.798728105Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=56.437µs 2024-09-17T11:43:02.840 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.799529551Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.803167909Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.638024ms 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.803549014Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.804197571Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=648.335µs 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.804572885Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.808188052Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.61481ms 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.808547619Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.80905287Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=505.215µs 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.80941351Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.810044569Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=630.88µs 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.810402927Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.814129324Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.726059ms 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.814489572Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T11:43:02.841 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.814996758Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=507.116µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.815362936Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.815996475Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=631.99µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.816341824Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.81684356Z level=info msg="Migration successfully executed" id="create alert_image table" duration=501.623µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.81723574Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.817900852Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=665.207µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.820196607Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.820284726Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=91.167µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.822981539Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.823648483Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=666.78µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.825406874Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.826060162Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=653.066µs 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.828502972Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T11:43:02.842 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.828759752Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.831194992Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.831525373Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=330.347µs 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.834050739Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.834795895Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=743.407µs 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.836755184Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.840503964Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.749246ms 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.840877618Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.841572174Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=702.222µs 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.843590373Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T11:43:03.079 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.844268204Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=677.399µs 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.846384672Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.846932488Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=547.612µs 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.849073956Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.849725517Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=651.312µs 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.852044144Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T11:43:03.080 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.852654628Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=610.179µs 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.854795405Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.854811241Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=16.481µs 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.857753139Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.857791196Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=38.376µs 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.860592824Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.860778606Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=185.695µs 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.863268252Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.863924866Z level=info msg="Migration successfully executed" id="create data_keys table" duration=656.302µs 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.865972406Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T11:43:03.081 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.866495329Z level=info msg="Migration successfully executed" id="create secrets table" duration=522.816µs 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.868836645Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.889710614Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=20.87343ms 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.890084398Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.894172942Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.088293ms 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.894530371Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.894622474Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=94.061µs 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.89725517Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.918190164Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.937223ms 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.918557383Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.939525899Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=20.968341ms 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.93990706Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.940458458Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=551.384µs 2024-09-17T11:43:03.082 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.942874882Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.943534008Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=658.863µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.945604011Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.945745192Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=141.128µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.948062799Z level=info msg="Executing migration" id="create permission table" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.948631124Z level=info msg="Migration successfully executed" id="create permission table" duration=568.249µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.951012951Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.95170608Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=695.312µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.953680679Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.954354701Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=673.64µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.956214818Z level=info msg="Executing migration" id="create role table" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.95678476Z level=info msg="Migration successfully executed" id="create role table" duration=568.403µs 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.958843951Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T11:43:03.083 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.963293712Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.44955ms 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.963646495Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.967785743Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.138903ms 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.968155903Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.968785483Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=627.903µs 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.970964543Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.971630216Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=665.4µs 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.973646943Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.974336075Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=704.592µs 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.976197307Z level=info msg="Executing migration" id="create team role table" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.976746849Z level=info msg="Migration successfully executed" id="create team role table" duration=549.235µs 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.977090591Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.977743724Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=651.886µs 2024-09-17T11:43:03.084 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.978263402Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.979112984Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=851.494µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.979519945Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.980154818Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=634.783µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.980504298Z level=info msg="Executing migration" id="create user role table" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.981035897Z level=info msg="Migration successfully executed" id="create user role table" duration=531.385µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.981419329Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.982053769Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=634.154µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.982383993Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.983028998Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=644.687µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.983391429Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.984011067Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=619.592µs 2024-09-17T11:43:03.085 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.98433998Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.98485968Z level=info msg="Migration successfully executed" id="create builtin role table" duration=519.467µs 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.985214377Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.98587824Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=654.674µs 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.986211717Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.986852347Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=640.543µs 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.987190999Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.991725029Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.533765ms 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.992072409Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.992692793Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=620.286µs 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.993051557Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.993687257Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=635.321µs 2024-09-17T11:43:03.086 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.994008825Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T11:43:03.087 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.994623999Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=615.17µs 2024-09-17T11:43:03.087 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.994948697Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.995569405Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=620.591µs 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.995895611Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.996420067Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=524.197µs 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.99675755Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.997398526Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=640.756µs 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:02 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:02.997744963Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.002408047Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.662811ms 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.002764657Z level=info msg="Executing migration" id="permission kind migration" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.007341433Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.5763ms 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.007690236Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.01215372Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.463375ms 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.012519886Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-17T11:43:03.088 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.017032129Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.511895ms 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.01736795Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.018030373Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=662.381µs 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.01836864Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.019070188Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=702.301µs 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.019404122Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.020034989Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=630.811µs 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.02036766Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.020932031Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=564.127µs 2024-09-17T11:43:03.089 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.021285614Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.021937845Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=652.039µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.022265602Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.02230362Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=38.222µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.022678731Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.022698719Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=20.621µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.023083657Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.023326537Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=242.883µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.023696428Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.024026236Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=330.322µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.02439599Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.024785325Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=389.437µs 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.025209778Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T11:43:03.090 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.025374979Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=165.633µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.025782955Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.026105038Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=321.902µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.026509511Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.02702816Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=518.528µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.027378163Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.028051508Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=673.248µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.028382434Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.03298153Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.598941ms 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.033322018Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.033360099Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=40.556µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.033771749Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.034457171Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=685.579µs 2024-09-17T11:43:03.091 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.034819869Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.035482488Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=661.358µs 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.035821812Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.036486427Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=664.422µs 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.036827745Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.041612941Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.785426ms 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.0419913Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.042647699Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=656.333µs 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.04299477Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.043643701Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=647.665µs 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.044003317Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.057166844Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=13.16373ms 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.057539339Z level=info msg="Executing migration" id="create correlation v2" 2024-09-17T11:43:03.092 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.058295441Z level=info msg="Migration successfully executed" id="create correlation v2" duration=755.97µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.058661396Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.059324922Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=663.434µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.059669118Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.060398957Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=730.251µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.060753414Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.06142152Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=668.424µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.06178243Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.061962888Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=179.87µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.062419106Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.062951477Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=532.109µs 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.064598562Z level=info msg="Executing migration" id="add provisioning column" 2024-09-17T11:43:03.093 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.069354956Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.756ms 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.069714949Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.070244012Z level=info msg="Migration successfully executed" id="create entity_events table" duration=529.039µs 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.072550121Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.073262028Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=711.73µs 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.075467184Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.075750019Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.077968091Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.078291665Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.078690411Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.079255587Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=564.903µs 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.079620808Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T11:43:03.094 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.080287547Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=666.529µs 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: Deploying daemon node-exporter.a on smithi049 2024-09-17T11:43:03.330 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-mon[33364]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.082496278Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.083290034Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=793.777µs 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.085215352Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.086029394Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=814.113µs 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.088015563Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.088653485Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=637.818µs 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.090673333Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.091349736Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=677.139µs 2024-09-17T11:43:03.331 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.093330417Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.093901155Z level=info msg="Migration successfully executed" id="Drop public config table" duration=568.584µs 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.096121279Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.096799786Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=678.285µs 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.098837793Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.099523166Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=684.146µs 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.101668166Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.102512858Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=846.729µs 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.104149721Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.104804686Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=654.858µs 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.106790926Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T11:43:03.332 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.122537407Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=15.746006ms 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.122916293Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.12793883Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.022388ms 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.128291051Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.13325438Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.96271ms 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.133598851Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.133751896Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=153.693µs 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.134139072Z level=info msg="Executing migration" id="add share column" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.139012289Z level=info msg="Migration successfully executed" id="add share column" duration=4.873324ms 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.139370342Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.139505203Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=134.959µs 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.141949334Z level=info msg="Executing migration" id="create file table" 2024-09-17T11:43:03.333 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.142548371Z level=info msg="Migration successfully executed" id="create file table" duration=598.771µs 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.142994874Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.143668638Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=673.732µs 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.145825449Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.146565602Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=743.989µs 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.14853997Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.149105875Z level=info msg="Migration successfully executed" id="create file_meta table" duration=569.617µs 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.151163265Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.151911443Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=751.406µs 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.152303732Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T11:43:03.334 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.1523535Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=50.461µs 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.155008802Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.155099045Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=91.498µs 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.157592305Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.157984806Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=396.049µs 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.159851541Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.159997449Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=145.48µs 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.160408455Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.16128134Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=873.322µs 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.163071634Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T11:43:03.335 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.167952928Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=4.881627ms 2024-09-17T11:43:03.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.168300911Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T11:43:03.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.168408465Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=107.412µs 2024-09-17T11:43:03.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.171013006Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T11:43:03.336 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.171766438Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=753.705µs 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.172300373Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.172766455Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=467.778µs 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.174984383Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.175120157Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=135.77µs 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.177230048Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.177543074Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=312.721µs 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.178012429Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.182849871Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=4.837222ms 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.183223887Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.188203378Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=4.979147ms 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.188563147Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.189253095Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=689.971µs 2024-09-17T11:43:03.337 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.189602156Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.245340941Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=55.73822ms 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.245707573Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.246436929Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=729.21µs 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.246780422Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.247510687Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=730.495µs 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.247889085Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.263852977Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=15.963707ms 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.26424939Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.269289252Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.040621ms 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.269648192Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.269840159Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=192.079µs 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.270273949Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-17T11:43:03.338 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.270382719Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=108.305µs 2024-09-17T11:43:03.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.271046307Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T11:43:03.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.271180981Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=134.807µs 2024-09-17T11:43:03.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.271906036Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-17T11:43:03.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.272035933Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=130.556µs 2024-09-17T11:43:03.340 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.272831352Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.273164454Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=336.713µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.273697328Z level=info msg="Executing migration" id="create folder table" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.274304191Z level=info msg="Migration successfully executed" id="create folder table" duration=607.18µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.274646018Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.275447942Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=801.708µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.275825106Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.276544965Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=719.992µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.276915935Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.276934506Z level=info msg="Migration successfully executed" id="Update folder title length" duration=19.622µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.277330323Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.278037452Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=706.942µs 2024-09-17T11:43:03.341 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.27838318Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.279044901Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=660.03µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.279379452Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.280110549Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=730.614µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.280465157Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.280753715Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=288.341µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.281081847Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.281254031Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=172.122µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.281910693Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.282560306Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=649.478µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.282900089Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.283588882Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=688.527µs 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.28392509Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-17T11:43:03.342 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.284572543Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=646.281µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.28490928Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.285631731Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=722.109µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.285976982Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.286650582Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=673.594µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.286997808Z level=info msg="Executing migration" id="create anon_device table" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.287547374Z level=info msg="Migration successfully executed" id="create anon_device table" duration=549.5µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.287921716Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.288667568Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=745.523µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.289038543Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.28973694Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=698.187µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.290102417Z level=info msg="Executing migration" id="create signing_key table" 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.290714691Z level=info msg="Migration successfully executed" id="create signing_key table" duration=612.149µs 2024-09-17T11:43:03.343 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.291069512Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.291735939Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=666.219µs 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.29208853Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.292758004Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=669.258µs 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.293097751Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.293262681Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=165.437µs 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.293936847Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.299562602Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.625432ms 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.299939554Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.300419368Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=480.425µs 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.3029239Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.303646169Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=721.756µs 2024-09-17T11:43:03.344 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.304001842Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.30468092Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=679.156µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.306954704Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.30772436Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=772.01µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.30940432Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.310163572Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=758.86µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.312314206Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.313173204Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=859.649µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.314842299Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.315510893Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=667.264µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.317786582Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.318379995Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=596.529µs 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.320493196Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-17T11:43:03.345 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.320743902Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=253.632µs 2024-09-17T11:43:03.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.321174568Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-17T11:43:03.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.32122209Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=48.224µs 2024-09-17T11:43:03.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.323863992Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-17T11:43:03.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.329229931Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.369635ms 2024-09-17T11:43:03.346 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.329651948Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-17T11:43:03.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: Deploying daemon node-exporter.a on smithi049 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[40223]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:03.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: Deploying daemon node-exporter.a on smithi049 2024-09-17T11:43:03.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:03 smithi049 ceph-mon[30017]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:03.402 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:03 smithi049 bash[74144]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-17T11:43:03.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.334972696Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.321052ms 2024-09-17T11:43:03.587 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.335306329Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.335528279Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=222.426µs 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=migrator t=2024-09-17T11:43:03.338082417Z level=info msg="migrations completed" performed=547 skipped=0 duration=1.250653913s 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore t=2024-09-17T11:43:03.339094935Z level=info msg="Created default organization" 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=secrets t=2024-09-17T11:43:03.341019334Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=plugin.store t=2024-09-17T11:43:03.358776197Z level=info msg="Loading plugins..." 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=local.finder t=2024-09-17T11:43:03.399500548Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=plugin.store t=2024-09-17T11:43:03.399517192Z level=info msg="Plugins loaded" count=55 duration=40.741653ms 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=query_data t=2024-09-17T11:43:03.401598976Z level=info msg="Query Service initialization" 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=live.push_http t=2024-09-17T11:43:03.40416674Z level=info msg="Live Push Gateway initialization" 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.migration t=2024-09-17T11:43:03.405280018Z level=info msg=Starting 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.migration t=2024-09-17T11:43:03.405568381Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.migration orgID=1 t=2024-09-17T11:43:03.40584718Z level=info msg="Migrating alerts for organisation" 2024-09-17T11:43:03.588 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.migration orgID=1 t=2024-09-17T11:43:03.406309713Z level=info msg="Alerts found to migrate" alerts=0 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.migration t=2024-09-17T11:43:03.407487596Z level=info msg="Completed alerting migration" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.state.manager t=2024-09-17T11:43:03.419201447Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=infra.usagestats.collector t=2024-09-17T11:43:03.420601487Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.datasources t=2024-09-17T11:43:03.422233746Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.datasources t=2024-09-17T11:43:03.434486581Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.alerting t=2024-09-17T11:43:03.447197619Z level=info msg="starting to provision alerting" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.alerting t=2024-09-17T11:43:03.447207727Z level=info msg="finished to provision alerting" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.state.manager t=2024-09-17T11:43:03.447394538Z level=info msg="Warming state cache for startup" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.multiorg.alertmanager t=2024-09-17T11:43:03.447608128Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafanaStorageLogger t=2024-09-17T11:43:03.447687458Z level=info msg="Storage starting" 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.state.manager t=2024-09-17T11:43:03.447700317Z level=info msg="State cache has been initialized" states=0 duration=305.282µs 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ngalert.scheduler t=2024-09-17T11:43:03.447803556Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-17T11:43:03.589 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ticker t=2024-09-17T11:43:03.447882815Z level=info msg=starting first_tick=2024-09-17T11:43:10Z 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=http.server t=2024-09-17T11:43:03.449687127Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=http.server t=2024-09-17T11:43:03.449923722Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.dashboard t=2024-09-17T11:43:03.466684996Z level=info msg="starting to provision dashboards" 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore.transactions t=2024-09-17T11:43:03.494476554Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=plugins.update.checker t=2024-09-17T11:43:03.537609983Z level=info msg="Update check succeeded" duration=90.200989ms 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafana.update.checker t=2024-09-17T11:43:03.53831046Z level=info msg="Update check succeeded" duration=90.903241ms 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore.transactions t=2024-09-17T11:43:03.56332919Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore.transactions t=2024-09-17T11:43:03.573823517Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-17T11:43:03.590 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore.transactions t=2024-09-17T11:43:03.586083117Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T11:43:03.926 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=provisioning.dashboard t=2024-09-17T11:43:03.641034402Z level=info msg="finished to provision dashboards" 2024-09-17T11:43:04.150 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:03 smithi049 bash[74144]: Getting image source signatures 2024-09-17T11:43:04.150 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:03 smithi049 bash[74144]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-17T11:43:04.150 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:03 smithi049 bash[74144]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-17T11:43:04.150 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:03 smithi049 bash[74144]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-17T11:43:04.426 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafana-apiserver t=2024-09-17T11:43:03.969142978Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-17T11:43:04.426 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:43:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafana-apiserver t=2024-09-17T11:43:03.969521078Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-17T11:43:05.400 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 bash[74144]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-17T11:43:05.400 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 bash[74144]: Writing manifest to image destination 2024-09-17T11:43:05.805 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-mon[40223]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:05.806 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-mon[30017]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:05.806 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 podman[74144]: 2024-09-17 11:43:05.457295085 +0000 UTC m=+2.301822114 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-17T11:43:05.806 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 podman[74144]: 2024-09-17 11:43:05.562816014 +0000 UTC m=+2.407343043 container create 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:05.806 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 podman[74144]: 2024-09-17 11:43:05.804903403 +0000 UTC m=+2.649430432 container init 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:05 smithi129 ceph-mon[33364]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:06.150 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 podman[74144]: 2024-09-17 11:43:05.811952869 +0000 UTC m=+2.656479898 container start 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.814Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.814Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.815Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.815Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.815Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.815Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T11:43:06.151 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T11:43:06.152 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T11:43:06.153 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T11:43:06.154 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a[74217]: ts=2024-09-17T11:43:05.816Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 bash[74144]: 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 2024-09-17T11:43:06.155 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 11:43:05 smithi049 systemd[1]: Started Ceph node-exporter.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:06 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:06 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:06 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:06 smithi129 ceph-mon[33364]: Deploying daemon node-exporter.b on smithi129 2024-09-17T11:43:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:06 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.177 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:07 smithi129 bash[63072]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[40223]: Deploying daemon node-exporter.b on smithi129 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[30017]: Deploying daemon node-exporter.b on smithi129 2024-09-17T11:43:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:06 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:07.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:07.677 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:07 smithi129 bash[63072]: Getting image source signatures 2024-09-17T11:43:07.677 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:07 smithi129 bash[63072]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-17T11:43:07.677 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:07 smithi129 bash[63072]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-17T11:43:07.677 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:07 smithi129 bash[63072]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-17T11:43:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:07 smithi049 ceph-mon[40223]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:07 smithi049 ceph-mon[30017]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:08.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:07 smithi129 ceph-mon[33364]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:09.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:09.426 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 bash[63072]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-17T11:43:09.426 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 bash[63072]: Writing manifest to image destination 2024-09-17T11:43:09.927 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 podman[63072]: 2024-09-17 11:43:09.618890574 +0000 UTC m=+2.710892214 container create e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T11:43:09.927 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 podman[63072]: 2024-09-17 11:43:09.536505824 +0000 UTC m=+2.628507407 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-17T11:43:09.927 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 podman[63072]: 2024-09-17 11:43:09.854072652 +0000 UTC m=+2.946074245 container init e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T11:43:09.927 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 podman[63072]: 2024-09-17 11:43:09.858775301 +0000 UTC m=+2.950776882 container start e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T11:43:09.927 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.861Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.861Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T11:43:09.928 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T11:43:09.929 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T11:43:09.930 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.862Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T11:43:09.931 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-17T11:43:09.932 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b[63149]: ts=2024-09-17T11:43:09.863Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[40223]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[30017]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:10 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.426 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 bash[63072]: e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 2024-09-17T11:43:10.426 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 11:43:09 smithi129 systemd[1]: Started Ceph node-exporter.b for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:10 smithi129 ceph-mon[33364]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:10 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:10 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:10 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:10 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:11.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:11 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:11 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:11 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:11 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:11 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:11 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[40223]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[30017]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:12 smithi129 ceph-mon[33364]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:12 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[40223]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:14.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[30017]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:14 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:14 smithi129 ceph-mon[33364]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:14 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:14 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:16.351 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:16 smithi129 ceph-mon[33364]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:16.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:16 smithi049 ceph-mon[40223]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:16.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:16 smithi049 ceph-mon[30017]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:17.613 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:18.309 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:18.309 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.309 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:18.310 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:18 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:18 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[40223]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[40223]: Reconfiguring daemon alertmanager.a on smithi049 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[30017]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T11:43:19.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:19 smithi049 ceph-mon[30017]: Reconfiguring daemon alertmanager.a on smithi049 2024-09-17T11:43:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:19 smithi129 ceph-mon[33364]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T11:43:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:19 smithi129 ceph-mon[33364]: Reconfiguring daemon alertmanager.a on smithi049 2024-09-17T11:43:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:20 smithi049 ceph-mon[40223]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:20.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:20 smithi049 ceph-mon[30017]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:20 smithi129 ceph-mon[33364]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:22.016 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:22 smithi049 ceph-mon[30017]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:22.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:22 smithi049 systemd[1]: Stopping Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:43:22.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:22 smithi049 ceph-mon[40223]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:22 smithi129 ceph-mon[33364]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:22.792 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:22 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[73904]: ts=2024-09-17T11:43:22.508Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T11:43:22.792 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:22 smithi049 podman[74934]: 2024-09-17 11:43:22.539460646 +0000 UTC m=+0.151800336 container died f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:23.150 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:22 smithi049 podman[74934]: 2024-09-17 11:43:22.963323176 +0000 UTC m=+0.575662858 container cleanup f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:23.150 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:22 smithi049 bash[74934]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T11:43:23.650 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 podman[74950]: 2024-09-17 11:43:23.181335201 +0000 UTC m=+0.644570884 container remove f4f8704fad1cba8db6e3b51b5aeae9dfceb89f2c078bbc23f26a4ede42fdbc8c (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:23.650 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 podman[74950]: 2024-09-17 11:43:23.282035254 +0000 UTC m=+0.745270936 volume remove e397af2d4bb75139ae8a75443e3dfb9322f52edbbd22f51750912c71630ab141 2024-09-17T11:43:24.018 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T11:43:24.018 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 systemd[1]: Stopped Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:24.018 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.102s CPU time. 2024-09-17T11:43:24.019 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 systemd[1]: Starting Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:43:24.019 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 podman[75037]: 2024-09-17 11:43:23.847012463 +0000 UTC m=+0.023669766 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-17T11:43:24.019 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:23 smithi049 podman[75037]: 2024-09-17 11:43:23.949595999 +0000 UTC m=+0.126253280 volume create 7456266d2b9bb247de954a468e1487e5d9debfbf2769c8480a0819c883928bde 2024-09-17T11:43:24.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-mon[30017]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:24.397 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-mon[40223]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:24.397 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 podman[75037]: 2024-09-17 11:43:24.033384566 +0000 UTC m=+0.210041855 container create 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:24.397 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 podman[75037]: 2024-09-17 11:43:24.395419077 +0000 UTC m=+0.572076380 container init 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:24 smithi129 ceph-mon[33364]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:43:24.650 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 podman[75037]: 2024-09-17 11:43:24.402469776 +0000 UTC m=+0.579127067 container start 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.418Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.418Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.419Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.49 port=9094 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.421Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.449Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.449Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.451Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:24.451Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 bash[75037]: 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 2024-09-17T11:43:24.651 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:24 smithi049 systemd[1]: Started Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:25.693 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:25 smithi129 ceph-mon[33364]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:25.693 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:25 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.693 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:25 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.693 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:25 smithi129 ceph-mon[33364]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T11:43:25.693 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:25 smithi129 ceph-mon[33364]: Reconfiguring daemon prometheus.a on smithi129 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[40223]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[40223]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[40223]: Reconfiguring daemon prometheus.a on smithi129 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[30017]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[30017]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T11:43:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:25 smithi049 ceph-mon[30017]: Reconfiguring daemon prometheus.a on smithi129 2024-09-17T11:43:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:26 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:26.421Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000562761s 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]} : dispatch 2024-09-17T11:43:26.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:26 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-17T11:43:27.865 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:27.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T11:43:27.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T11:43:27.866 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:27.866 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:27.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:27.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T11:43:27.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-17T11:43:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T11:43:27.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:28.176 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 systemd[1]: Stopping Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:43:28.575 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T11:43:28.575 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-17T11:43:28.575 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-17T11:43:28.575 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T11:43:28.575 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.295Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.296Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.296Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[61236]: ts=2024-09-17T11:43:28.296Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-17T11:43:28.576 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 podman[63986]: 2024-09-17 11:43:28.305682667 +0000 UTC m=+0.122863430 container died 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:28 smithi049 ceph-mon[30017]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T11:43:28.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:28 smithi049 ceph-mon[40223]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T11:43:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:28 smithi129 ceph-mon[33364]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T11:43:28.926 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 podman[63986]: 2024-09-17 11:43:28.7017523 +0000 UTC m=+0.518933061 container cleanup 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:28.927 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 bash[63986]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T11:43:29.330 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:28 smithi129 podman[64003]: 2024-09-17 11:43:28.948473861 +0000 UTC m=+0.642791004 container remove 6922f96d8228f6791be609526cd039e6192048c50beb0fccb0310eeb9a622f0a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:29.330 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-17T11:43:29.330 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 systemd[1]: Stopped Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:29.331 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.098s CPU time. 2024-09-17T11:43:29.331 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 systemd[1]: Starting Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T11:43:29.582 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-mon[33364]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:43:29.582 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-mon[33364]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-17T11:43:29.583 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 podman[64102]: 2024-09-17 11:43:29.233304573 +0000 UTC m=+0.016365946 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-17T11:43:29.583 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 podman[64102]: 2024-09-17 11:43:29.354478698 +0000 UTC m=+0.137540055 container create e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:29.583 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 podman[64102]: 2024-09-17 11:43:29.581730048 +0000 UTC m=+0.364791411 container init e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:29.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-mon[30017]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:43:29.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-mon[30017]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-17T11:43:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-mon[40223]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:43:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-mon[40223]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-17T11:43:29.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Bus STOPPING 2024-09-17T11:43:29.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T11:43:29.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Bus STOPPED 2024-09-17T11:43:29.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Bus STARTING 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 podman[64102]: 2024-09-17 11:43:29.586631997 +0000 UTC m=+0.369693358 container start e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.606Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.606Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.606Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi129 (none))" 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.606Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T11:43:29.929 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.606Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.608Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.608Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.610Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.610Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.611Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.611Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.103µs 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.611Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.612Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.612Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-17T11:43:29.930 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.612Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=35.77µs wal_replay_duration=433.302µs wbl_replay_duration=90ns total_replay_duration=490.292µs 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.614Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.614Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.614Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.629Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.644723ms db_storage=775ns remote_storage=1.07µs web_handler=302ns query_engine=511ns scrape=375.651µs scrape_sd=418.31µs notify=23.238µs notify_sd=17.04µs rules=14.484011ms tracing=6.861µs 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.629Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T11:43:29.629Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 bash[64102]: e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 2024-09-17T11:43:29.931 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 11:43:29 smithi129 systemd[1]: Started Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T11:43:30.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Serving on http://:::9283 2024-09-17T11:43:30.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Bus STARTED 2024-09-17T11:43:30.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:29 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:29] ENGINE Bus STOPPING 2024-09-17T11:43:30.726 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE Bus STOPPED 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE Bus STARTING 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE Serving on http://:::9283 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE Bus STARTED 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:30] ENGINE Bus STOPPING 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"} : dispatch 2024-09-17T11:43:30.727 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"} : dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"} : dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:30.728 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:30.729 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:30.729 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:31.020 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.020 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"} : dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"} : dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T11:43:31.021 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"} : dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:30 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-17T11:43:31.022 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"} : dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi049.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"} : dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi129.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"} : dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi129.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T11:43:31.023 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:31.024 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:31.024 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:31.024 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:43:31.024 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:30 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:31.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T11:43:31.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:31] ENGINE Bus STOPPED 2024-09-17T11:43:31.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:31] ENGINE Bus STARTING 2024-09-17T11:43:31.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:31] ENGINE Serving on http://:::9283 2024-09-17T11:43:31.400 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:11:43:31] ENGINE Bus STARTED 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[40223]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[30017]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:43:32.071 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:32.072 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:31 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:31 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:31 smithi129 ceph-mon[33364]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:43:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:31 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-17T11:43:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:31 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-17T11:43:33.109 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:32 smithi049 ceph-mon[40223]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-17T11:43:33.109 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:32 smithi049 ceph-mon[40223]: Cluster is now healthy 2024-09-17T11:43:33.110 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:32 smithi049 ceph-mon[30017]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-17T11:43:33.110 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:32 smithi049 ceph-mon[30017]: Cluster is now healthy 2024-09-17T11:43:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:32 smithi129 ceph-mon[33364]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-17T11:43:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:32 smithi129 ceph-mon[33364]: Cluster is now healthy 2024-09-17T11:43:34.101 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:33 smithi129 ceph-mon[33364]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:34.102 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:33 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.102 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:33 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[40223]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[30017]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:33 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:34.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 11:43:34 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T11:43:34.425Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004025866s 2024-09-17T11:43:35.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:35 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:43:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:35 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:37.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:37 smithi049 ceph-mon[40223]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 55 B/s, 1 objects/s recovering 2024-09-17T11:43:37.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:37 smithi049 ceph-mon[30017]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 55 B/s, 1 objects/s recovering 2024-09-17T11:43:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:37 smithi129 ceph-mon[33364]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 55 B/s, 1 objects/s recovering 2024-09-17T11:43:37.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:38.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:38.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:38.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:39.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:39 smithi049 ceph-mon[40223]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s; 50 B/s, 1 objects/s recovering 2024-09-17T11:43:39.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:39 smithi049 ceph-mon[30017]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s; 50 B/s, 1 objects/s recovering 2024-09-17T11:43:39.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:39 smithi129 ceph-mon[33364]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s; 50 B/s, 1 objects/s recovering 2024-09-17T11:43:41.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:41 smithi049 ceph-mon[40223]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:41 smithi049 ceph-mon[30017]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:41.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:41 smithi129 ceph-mon[33364]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:42.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:42.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:42.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:43.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:43 smithi049 ceph-mon[30017]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:43 smithi049 ceph-mon[40223]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:43 smithi129 ceph-mon[33364]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 45 B/s, 0 objects/s recovering 2024-09-17T11:43:45.691 INFO:tasks.workunit.client.0.smithi049.stderr:Updating files: 96% (12457/12893) Updating files: 97% (12507/12893) Updating files: 98% (12636/12893) Updating files: 99% (12765/12893) Updating files: 100% (12893/12893) Updating files: 100% (12893/12893), done. 2024-09-17T11:43:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:45 smithi049 ceph-mon[40223]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:45.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:45 smithi049 ceph-mon[30017]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:45 smithi129 ceph-mon[33364]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:46.261 INFO:tasks.workunit.client.0.smithi049.stderr:Note: switching to '2b698643d05f73e79c2d2888228947a84a4ac0bd'. 2024-09-17T11:43:46.261 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.261 INFO:tasks.workunit.client.0.smithi049.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:state without impacting any branches by switching back to a branch. 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: git switch -c 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:Or undo this operation with: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: git switch - 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-17T11:43:46.262 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-09-17T11:43:46.263 INFO:tasks.workunit.client.0.smithi049.stderr:HEAD is now at 2b698643d05 Merge pull request #59781 from ivoalmeida/table-filter-fix 2024-09-17T11:43:46.264 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-17T11:43:46.295 INFO:tasks.workunit.client.0.smithi049.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-17T11:43:46.296 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-17T11:43:46.297 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-17T11:43:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:46.954 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-17T11:43:47.016 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-17T11:43:47.057 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-17T11:43:47.059 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-17T11:43:47.059 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-17T11:43:47.100 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-17T11:43:47.104 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-09-17T11:43:47.104 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-17T11:43:47.161 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-17T11:43:47.162 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-17T11:43:47.162 DEBUG:teuthology.orchestra.run.smithi049:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2b698643d05f73e79c2d2888228947a84a4ac0bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-17T11:43:47.223 INFO:tasks.workunit.client.0.smithi049.stderr:+ parallel=1 2024-09-17T11:43:47.223 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' '' = --serial ']' 2024-09-17T11:43:47.223 INFO:tasks.workunit.client.0.smithi049.stderr:+ crimson=0 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' '' = --crimson ']' 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ color= 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -t 1 ']' 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-17T11:43:47.224 INFO:tasks.workunit.client.0.smithi049.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:+ declare -A pids 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_aio 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_aio' 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_aio 2024-09-17T11:43:47.225 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_aio 2024-09-17T11:43:47.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75654 2024-09-17T11:43:47.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_aio on pid 75654' 2024-09-17T11:43:47.227 INFO:tasks.workunit.client.0.smithi049.stdout:test api_aio on pid 75654 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75654 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_aio_pp 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_aio_pp' 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_aio_pp 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.231 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_aio_pp 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75661 2024-09-17T11:43:47.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_aio_pp on pid 75661' 2024-09-17T11:43:47.233 INFO:tasks.workunit.client.0.smithi049.stdout:test api_aio_pp on pid 75661 2024-09-17T11:43:47.236 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75661 2024-09-17T11:43:47.236 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.236 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.236 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-17T11:43:47.236 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_io 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_io' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_io 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.237 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.238 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.239 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_io 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75670 2024-09-17T11:43:47.240 INFO:tasks.workunit.client.0.smithi049.stdout:test api_io on pid 75670 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_io on pid 75670' 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75670 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.244 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_io_pp 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_io_pp' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_io_pp 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.245 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.246 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.247 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.248 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_io_pp 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75682 2024-09-17T11:43:47.249 INFO:tasks.workunit.client.0.smithi049.stdout:test api_io_pp on pid 75682 2024-09-17T11:43:47.253 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_io_pp on pid 75682' 2024-09-17T11:43:47.253 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75682 2024-09-17T11:43:47.253 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_asio 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_asio' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_asio 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.254 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.255 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.256 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.257 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.258 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_asio 2024-09-17T11:43:47.259 INFO:tasks.workunit.client.0.smithi049.stdout:test api_asio on pid 75696 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75696 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_asio on pid 75696' 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75696 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-17T11:43:47.265 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_list 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_list' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.266 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.267 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.268 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_list 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.269 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.270 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.271 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.272 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75654/exe 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.273 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_list 2024-09-17T11:43:47.274 INFO:tasks.workunit.client.0.smithi049.stdout:test api_list on pid 75714 2024-09-17T11:43:47.275 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75714 2024-09-17T11:43:47.275 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.275 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_list on pid 75714' 2024-09-17T11:43:47.275 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75714 2024-09-17T11:43:47.275 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_lock 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_lock' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.276 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.277 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.278 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.279 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.280 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.281 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_aio: /' 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75661/exe 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.282 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_lock 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.283 INFO:tasks.workunit.client.0.smithi049.stdout:test api_lock on pid 75733 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_lock 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75733 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_lock on pid 75733' 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75733 2024-09-17T11:43:47.286 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_lock_pp 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.287 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_lock_pp' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.288 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.289 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.290 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.291 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75670/exe 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.292 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.293 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_lock_pp 2024-09-17T11:43:47.294 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_io.log 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_lock_pp 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.295 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.296 INFO:tasks.workunit.client.0.smithi049.stdout:test api_lock_pp on pid 75756 2024-09-17T11:43:47.300 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.300 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75756 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_lock_pp on pid 75756' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75756 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.301 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_io: /' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.302 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.303 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.304 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_misc 2024-09-17T11:43:47.305 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_misc' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.306 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.307 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.308 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.309 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.310 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_misc 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.311 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.312 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.313 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.314 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.314 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.314 INFO:tasks.workunit.client.0.smithi049.stdout:test api_misc on pid 75780 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_misc 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.319 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75780 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_misc on pid 75780' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75780 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.320 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75682/exe 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.321 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75696/exe 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.322 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.323 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.324 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.325 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_asio: /' 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_misc_pp 2024-09-17T11:43:47.326 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_misc_pp' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_misc_pp 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.327 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75714/exe 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.328 INFO:tasks.workunit.client.0.smithi049.stdout:test api_misc_pp on pid 75802 2024-09-17T11:43:47.330 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_misc_pp 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75802 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_misc_pp on pid 75802' 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75802 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.331 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_list: /' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_list.log 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_tier_pp 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_tier_pp' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.333 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.334 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.335 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.336 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_tier_pp 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.337 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_tier_pp 2024-09-17T11:43:47.338 INFO:tasks.workunit.client.0.smithi049.stdout:test api_tier_pp on pid 75833 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75833 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_tier_pp on pid 75833' 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75833 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_pool 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_pool' 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.342 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.343 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75733/exe 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.344 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.345 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.346 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.347 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.348 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_lock: /' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_pool 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.349 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.350 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.351 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.351 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.351 INFO:tasks.workunit.client.0.smithi049.stdout:test api_pool on pid 75891 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_pool 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75891 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_pool on pid 75891' 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75891 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.357 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.358 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.359 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.360 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75756/exe 2024-09-17T11:43:47.361 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_snapshots 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_snapshots' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.362 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.363 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.364 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_snapshots 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.365 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.366 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.367 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.368 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.369 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.370 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.371 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.371 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.371 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.371 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.371 INFO:tasks.workunit.client.0.smithi049.stdout:test api_snapshots on pid 75951 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_snapshots 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75951 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_snapshots on pid 75951' 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75951 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.374 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.375 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75802/exe 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_snapshots_pp 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.376 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_snapshots_pp' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.377 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_snapshots_pp 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.378 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.379 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.380 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.381 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.382 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75780/exe 2024-09-17T11:43:47.383 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_snapshots_pp 2024-09-17T11:43:47.384 INFO:tasks.workunit.client.0.smithi049.stdout:test api_snapshots_pp on pid 75980 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75980 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_snapshots_pp on pid 75980' 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75980 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_stat 2024-09-17T11:43:47.385 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_stat' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.386 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_stat 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.387 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.388 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_misc: /' 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.389 INFO:tasks.workunit.client.0.smithi049.stdout:test api_stat on pid 75995 2024-09-17T11:43:47.393 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.393 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_stat 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75995 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_stat on pid 75995' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=75995 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.394 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75833/exe 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.395 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.396 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75891/exe 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.397 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_stat_pp 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_stat_pp' 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.398 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.399 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.400 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_stat_pp 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-09-17T11:43:47.401 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_pool: /' 2024-09-17T11:43:47.402 INFO:tasks.workunit.client.0.smithi049.stdout:test api_stat_pp on pid 76022 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_stat_pp 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76022 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_stat_pp on pid 76022' 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76022 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.404 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_watch_notify 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_watch_notify' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.405 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.406 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.407 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_watch_notify 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.408 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.409 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.410 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.410 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.410 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.410 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.410 INFO:tasks.workunit.client.0.smithi049.stdout:test api_watch_notify on pid 76057 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.415 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_watch_notify 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76057 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_watch_notify on pid 76057' 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76057 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.416 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.417 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-17T11:43:47.418 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.419 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_watch_notify_pp 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_watch_notify_pp' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75951/exe 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.420 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.421 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.422 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_watch_notify_pp 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.423 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.424 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.425 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.426 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75980/exe 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_watch_notify_pp 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76106 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_watch_notify_pp on pid 76106' 2024-09-17T11:43:47.427 INFO:tasks.workunit.client.0.smithi049.stdout:test api_watch_notify_pp on pid 76106 2024-09-17T11:43:47.430 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76106 2024-09-17T11:43:47.430 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.430 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.430 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.430 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_cmd 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.431 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_cmd' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.433 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_cmd 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.434 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.435 INFO:tasks.workunit.client.0.smithi049.stdout:test api_cmd on pid 76120 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_cmd 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.441 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76120 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_cmd on pid 76120' 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76120 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.442 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_cmd_pp 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_cmd_pp' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.443 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.444 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.445 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.446 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.447 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.448 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.449 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.450 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.451 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76022/exe 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/75995/exe 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.452 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.453 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.454 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.455 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.456 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_stat: /' 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.457 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.458 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76057/exe 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.459 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.460 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_cmd_pp 2024-09-17T11:43:47.461 INFO:tasks.workunit.client.0.smithi049.stdout:test api_cmd_pp on pid 76205 2024-09-17T11:43:47.464 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.464 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.464 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.464 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_cmd_pp 2024-09-17T11:43:47.464 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76205 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_cmd_pp on pid 76205' 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76205 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.465 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.466 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.467 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.468 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.469 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_service 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_service' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_service 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.470 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.471 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.472 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.472 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76120/exe 2024-09-17T11:43:47.472 INFO:tasks.workunit.client.0.smithi049.stdout:test api_service on pid 76228 2024-09-17T11:43:47.473 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_service 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76228 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_service on pid 76228' 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76228 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_service_pp 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_service_pp' 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.474 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_service_pp 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.475 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_cmd: /' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.476 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.477 INFO:tasks.workunit.client.0.smithi049.stdout:test api_service_pp on pid 76254 2024-09-17T11:43:47.479 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_service_pp 2024-09-17T11:43:47.479 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.479 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76254 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_service_pp on pid 76254' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76254 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.480 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.481 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.482 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76106/exe 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.483 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.484 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.485 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_c_write_operations 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_c_write_operations' 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.486 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.487 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.488 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_c_write_operations 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.494 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.495 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_c_write_operations 2024-09-17T11:43:47.496 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76333 2024-09-17T11:43:47.497 INFO:tasks.workunit.client.0.smithi049.stdout:test api_c_write_operations on pid 76333 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_c_write_operations on pid 76333' 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76333 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.498 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.499 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.500 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.500 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.500 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s api_c_read_operations 2024-09-17T11:43:47.500 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' api_c_read_operations' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.501 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.502 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo api_c_read_operations 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.506 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.507 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.508 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.509 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.509 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.510 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.511 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=api_c_read_operations 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.512 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76380 2024-09-17T11:43:47.513 INFO:tasks.workunit.client.0.smithi049.stdout:test api_c_read_operations on pid 76380 2024-09-17T11:43:47.515 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test api_c_read_operations on pid 76380' 2024-09-17T11:43:47.515 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76380 2024-09-17T11:43:47.515 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.515 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s list_parallel 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' list_parallel' 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.516 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.517 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76254/exe 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo list_parallel 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.518 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.519 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.520 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.520 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.520 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.521 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.522 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.523 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.523 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.523 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-09-17T11:43:47.524 INFO:tasks.workunit.client.0.smithi049.stdout:test list_parallel on pid 76408 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=list_parallel 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76408 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test list_parallel on pid 76408' 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.528 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76408 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.529 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s open_pools_parallel 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' open_pools_parallel' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo open_pools_parallel 2024-09-17T11:43:47.530 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76228/exe 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76205/exe 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.531 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.532 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.533 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=open_pools_parallel 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.534 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76425 2024-09-17T11:43:47.535 INFO:tasks.workunit.client.0.smithi049.stdout:test open_pools_parallel on pid 76425 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test open_pools_parallel on pid 76425' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76425 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.542 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_service: /' 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s delete_pools_parallel 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.543 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' delete_pools_parallel' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.544 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.545 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.546 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo delete_pools_parallel 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.547 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.548 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_service.log 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.549 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76333/exe 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.550 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.551 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.552 INFO:tasks.workunit.client.0.smithi049.stdout:test delete_pools_parallel on pid 76456 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=delete_pools_parallel 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.555 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76456 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test delete_pools_parallel on pid 76456' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76456 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.556 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.557 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s cls 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' cls' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo cls 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.558 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.559 INFO:tasks.workunit.client.0.smithi049.stdout:test cls on pid 76472 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=cls 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76472 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test cls on pid 76472' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76472 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.562 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.563 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.564 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.565 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s cmd 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.566 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.567 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' cmd' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.568 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.569 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.570 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.570 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.570 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.571 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.571 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.571 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.572 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.573 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.574 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.575 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.576 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.577 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo cmd 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.578 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.579 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.579 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.579 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.580 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.581 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.582 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.583 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.584 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.586 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.587 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.588 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76408/exe 2024-09-17T11:43:47.588 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76472/exe 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.589 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.590 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.591 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ cls: /' 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=cmd 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stdout:test cmd on pid 76547 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76547 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test cmd on pid 76547' 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76547 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.593 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_cls.log 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.594 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_cls 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s handler_error 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76380/exe 2024-09-17T11:43:47.595 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' handler_error' 2024-09-17T11:43:47.596 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.596 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.596 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.597 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.597 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.597 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.597 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.598 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.599 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo handler_error 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.600 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.601 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.601 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-09-17T11:43:47.601 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-09-17T11:43:47.602 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ list_parallel: /' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.605 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.606 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.608 INFO:tasks.workunit.client.0.smithi049.stdout:test handler_error on pid 76572 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=handler_error 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76572 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test handler_error on pid 76572' 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76572 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-09-17T11:43:47.609 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s io 2024-09-17T11:43:47.610 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' io' 2024-09-17T11:43:47.610 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.612 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.612 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.612 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.612 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.613 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.613 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.613 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo io 2024-09-17T11:43:47.613 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=io 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76579 2024-09-17T11:43:47.614 INFO:tasks.workunit.client.0.smithi049.stdout:test io on pid 76579 2024-09-17T11:43:47.615 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test io on pid 76579' 2024-09-17T11:43:47.615 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76579 2024-09-17T11:43:47.615 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.615 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.616 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-09-17T11:43:47.616 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.616 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.616 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.616 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.619 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.620 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76425/exe 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.621 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.622 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76456/exe 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.623 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s ec_io 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.624 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' ec_io' 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.625 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.626 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.629 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.630 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo ec_io 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.631 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.632 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.633 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.633 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.634 INFO:tasks.workunit.client.0.smithi049.stdout:test ec_io on pid 76618 2024-09-17T11:43:47.634 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=ec_io 2024-09-17T11:43:47.634 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76618 2024-09-17T11:43:47.634 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test ec_io on pid 76618' 2024-09-17T11:43:47.634 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76618 2024-09-17T11:43:47.635 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.635 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.635 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s list 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' list' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.636 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.637 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.637 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.637 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.638 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.639 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.640 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.641 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-17T11:43:47.642 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.644 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.645 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.648 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.649 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:47 smithi049 ceph-mon[30017]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:47.651 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:47 smithi049 ceph-mon[40223]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.658 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.659 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.660 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.661 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.661 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.661 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.661 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.661 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.662 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo list 2024-09-17T11:43:47.663 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.663 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.664 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.665 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.665 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.665 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.665 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.669 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.669 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.669 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.669 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.669 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.673 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.674 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.675 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.675 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.675 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.675 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.675 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.677 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.677 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.679 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.680 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.681 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76572/exe 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.683 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.684 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.685 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.685 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.685 INFO:tasks.workunit.client.0.smithi049.stdout:test list on pid 76696 2024-09-17T11:43:47.685 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=list 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76696 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test list on pid 76696' 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76696 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s ec_list 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.686 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.687 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.692 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.692 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.692 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.692 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.692 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' ec_list' 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo ec_list 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.693 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.694 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.694 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_handler_error 2024-09-17T11:43:47.694 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_handler_error.log 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.695 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=ec_list 2024-09-17T11:43:47.696 INFO:tasks.workunit.client.0.smithi049.stdout:test ec_list on pid 76714 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76714 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test ec_list on pid 76714' 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76714 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.697 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ handler_error: /' 2024-09-17T11:43:47.698 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76547/exe 2024-09-17T11:43:47.699 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.699 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.699 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.702 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.702 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.702 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s misc 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' misc' 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-09-17T11:43:47.703 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.704 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.704 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_cmd 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_cmd.log 2024-09-17T11:43:47.705 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ cmd: /' 2024-09-17T11:43:47.707 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.707 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.707 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.707 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.708 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.709 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.711 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76618/exe 2024-09-17T11:43:47.713 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.715 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.716 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.716 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ ec_io: /' 2024-09-17T11:43:47.716 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.721 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.722 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_ec_io 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_ec_io.log 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.723 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.724 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.728 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo misc 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stdout:test misc on pid 76791 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=misc 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76791 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test misc on pid 76791' 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76791 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.731 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.732 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.736 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.736 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.737 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s pool 2024-09-17T11:43:47.738 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-09-17T11:43:47.739 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' pool' 2024-09-17T11:43:47.746 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.747 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.748 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.749 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.750 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.750 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.750 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.751 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.751 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.751 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.751 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.754 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo pool 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.755 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.756 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.758 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.759 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.760 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.761 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.761 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.762 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.762 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.762 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.762 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.762 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.763 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.764 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.765 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76714/exe 2024-09-17T11:43:47.766 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.767 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.768 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ ec_list: /' 2024-09-17T11:43:47.768 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_ec_list.log 2024-09-17T11:43:47.770 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_ec_list 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.771 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.772 INFO:tasks.workunit.client.0.smithi049.stdout:test pool on pid 76888 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=pool 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76888 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test pool on pid 76888' 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76888 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.773 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.777 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.780 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76579/exe 2024-09-17T11:43:47.780 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-09-17T11:43:47.781 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s read_operations 2024-09-17T11:43:47.781 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.781 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.782 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.782 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.782 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.782 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' read_operations' 2024-09-17T11:43:47.782 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.783 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.785 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.785 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.786 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.787 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.788 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.789 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.790 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.790 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.790 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.790 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.791 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.792 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.793 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.794 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ io: /' 2024-09-17T11:43:47.794 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_io 2024-09-17T11:43:47.795 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_io.log 2024-09-17T11:43:47.796 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo read_operations 2024-09-17T11:43:47.797 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.799 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.800 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.801 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.801 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.801 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.801 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.801 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.802 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.806 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.807 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.808 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76696/exe 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.809 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.811 INFO:tasks.workunit.client.0.smithi049.stdout:test read_operations on pid 76966 2024-09-17T11:43:47.811 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.811 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.811 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.811 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=read_operations 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76966 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test read_operations on pid 76966' 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76966 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.812 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_list 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s snapshots 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' snapshots' 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_list.log 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.813 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.814 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo snapshots 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.817 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.818 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.819 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.820 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.820 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ list: /' 2024-09-17T11:43:47.820 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.820 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76888/exe 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stdout:test snapshots on pid 76993 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=snapshots 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.821 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76993 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test snapshots on pid 76993' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=76993 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.822 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.823 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76791/exe 2024-09-17T11:43:47.824 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.824 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s watch_notify 2024-09-17T11:43:47.824 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' watch_notify' 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.827 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.828 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.829 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo watch_notify 2024-09-17T11:43:47.829 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.829 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.829 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.829 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.830 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ pool: /' 2024-09-17T11:43:47.831 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.831 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_pool.log 2024-09-17T11:43:47.831 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.832 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ misc: /' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.833 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.834 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_misc 2024-09-17T11:43:47.835 INFO:tasks.workunit.client.0.smithi049.stdout:test watch_notify on pid 77022 2024-09-17T11:43:47.835 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_pool 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=watch_notify 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_misc.log 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=77022 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test watch_notify on pid 77022' 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=77022 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:++ printf %25s write_operations 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:+ r=' write_operations' 2024-09-17T11:43:47.836 INFO:tasks.workunit.client.0.smithi049.stderr:++ awk '{print $1}' 2024-09-17T11:43:47.837 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-09-17T11:43:47.837 INFO:tasks.workunit.client.0.smithi049.stderr:++ echo write_operations 2024-09-17T11:43:47.839 INFO:tasks.workunit.client.0.smithi049.stdout:test write_operations on pid 77067 2024-09-17T11:43:47.839 INFO:tasks.workunit.client.0.smithi049.stderr:+ ff=write_operations 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=77067 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo 'test write_operations on pid 77067' 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ pids[$f]=77067 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ ret=0 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75833 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75833 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.840 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.841 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.842 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.843 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.843 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.843 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.843 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.843 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.844 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.845 INFO:tasks.workunit.client.0.smithi049.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -f /etc/bashrc ']' 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:+ . /etc/bashrc 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -z '' ']' 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:++ BASHRCSOURCED=Y 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.847 INFO:tasks.workunit.client.0.smithi049.stderr:++ shopt -q login_shell 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.848 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.849 INFO:tasks.workunit.client.0.smithi049.stderr:+++ umask 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' 0022 -eq 0 ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ SHELL=/bin/bash 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-17T11:43:47.850 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.851 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.852 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.854 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.855 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.856 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.857 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.858 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.858 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.858 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.858 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.858 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76966/exe 2024-09-17T11:43:47.860 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'grep=grep --color=auto' 2024-09-17T11:43:47.860 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-17T11:43:47.860 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-17T11:43:47.860 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorls.sh 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' '!' -t 0 ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:+++ return 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export S_COLORS=auto 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:+++ S_COLORS=auto 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-17T11:43:47.861 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.862 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.863 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.864 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ read_operations: /' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_read_operations.log 2024-09-17T11:43:47.865 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-17T11:43:47.866 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_read_operations 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.869 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.870 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/76993/exe 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.871 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-17T11:43:47.872 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/gawk.sh 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/lang.sh 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup 2024-09-17T11:43:47.873 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ snapshots: /' 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_snapshots.log 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.874 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_snapshots 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.877 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/locale 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/77022/exe 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.878 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.879 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.880 INFO:tasks.workunit.client.0.smithi049.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-17T11:43:47.882 INFO:tasks.workunit.client.0.smithi049.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-17T11:43:47.882 INFO:tasks.workunit.client.0.smithi049.stderr:++++ export LANG=en_US.UTF-8 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:++++ LANG=en_US.UTF-8 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ LANG=en_US.UTF-8 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ unset LANG_backup config 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n '' ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' dumb = linux ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/less.sh 2024-09-17T11:43:47.883 INFO:tasks.workunit.client.0.smithi049.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-17T11:43:47.884 INFO:tasks.workunit.client.0.smithi049.stderr:++ for i in /etc/profile.d/*.sh 2024-09-17T11:43:47.884 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-17T11:43:47.884 INFO:tasks.workunit.client.0.smithi049.stderr:++ '[' '' ']' 2024-09-17T11:43:47.884 INFO:tasks.workunit.client.0.smithi049.stderr:++ . /etc/profile.d/which2.sh 2024-09-17T11:43:47.884 INFO:tasks.workunit.client.0.smithi049.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-17T11:43:47.885 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_watch_notify 2024-09-17T11:43:47.885 INFO:tasks.workunit.client.0.smithi049.stderr:+++++ readlink /proc/77067/exe 2024-09-17T11:43:47.885 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-09-17T11:43:47.886 INFO:tasks.workunit.client.0.smithi049.stderr:++++ basename /usr/bin/bash 2024-09-17T11:43:47.886 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ watch_notify: /' 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_declare='declare -f' 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+++ which_opt=-f 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export which_declare 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+++ export -f which 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset i 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:++ unset -f pathmunge 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ export PATH 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ unset rc 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ tee ceph_test_neorados_write_operations.log 2024-09-17T11:43:47.887 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph_test_neorados_write_operations 2024-09-17T11:43:47.888 INFO:tasks.workunit.client.0.smithi049.stderr:+ sed 's/^/ write_operations: /' 2024-09-17T11:43:47.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:47 smithi129 ceph-mon[33364]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:47.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1106616926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1630463448' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1087995178' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1882637256' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/661719264' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3608889147' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/959387118' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1381944367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3184513237' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2062032819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/545836333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1106616926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1630463448' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1087995178' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1882637256' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/661719264' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3608889147' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/959387118' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1381944367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3184513237' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2062032819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/545836333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.657 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.657 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.657 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.657 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.657 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.825 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [----------] Global test environment set-up. 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (11 ms) 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (4 ms) 2024-09-17T11:43:48.826 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (6 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (5 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-09-17T11:43:48.827 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [----------] 12 tests from AsioRados (40 ms total) 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [----------] Global test environment tear-down 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (1497 ms total) 2024-09-17T11:43:48.828 INFO:tasks.workunit.client.0.smithi049.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1106616926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1630463448' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1087995178' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1882637256' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/661719264' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3608889147' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/959387118' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1381944367' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3184513237' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2062032819' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/545836333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:48.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] Global test environment set-up. 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.ListObjects (246 ms) 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (27 ms) 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-17T11:43:49.073 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo1,foo2,foo3 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo1 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo2 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo3 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo1,foo4,foo5 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo4 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo5 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo1 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo6,foo7 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo7 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo6 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo4 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo5 2024-09-17T11:43:49.074 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns2:foo7 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns2:foo6 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo1 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo1 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo2 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo3 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (65 ms) 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 1 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 10 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 13 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 7 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 14 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 0 0 2024-09-17T11:43:49.075 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 15 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 11 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 5 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 8 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 6 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 3 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 4 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 12 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 9 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 2 0 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (63 ms) 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: x cursor=MIN 2024-09-17T11:43:49.076 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=1 cursor=12:02547ec2:::1:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=10 cursor=12:52ea6a34:::10:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=13 cursor=12:566253c9:::13:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=7 cursor=12:5c6b0b28:::7:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=14 cursor=12:62a1935d:::14:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=0 cursor=12:6cac518f:::0:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=15 cursor=12:863748b0:::15:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=11 cursor=12:89d3ae78:::11:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=5 cursor=12:b29083e3:::5:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=8 cursor=12:bd63b0f1:::8:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=6 cursor=12:c4fdafeb:::6:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=3 cursor=12:cfc208b3:::3:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=4 cursor=12:d83876eb:::4:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=12 cursor=12:de5d7c5f:::12:head 2024-09-17T11:43:49.077 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=9 cursor=12:e960b815:::9:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > oid=2 cursor=12:f905c69b:::2:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=1 cursor=12:02547ec2:::1:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=10 cursor=12:52ea6a34:::10:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=13 cursor=12:566253c9:::13:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:566253c9:::13:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=7 cursor=12:5c6b0b28:::7:head 2024-09-17T11:43:49.078 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-09-17T11:43:49.079 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-09-17T11:43:49.121 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=14 cursor=12:62a1935d:::14:head 2024-09-17T11:43:49.121 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-09-17T11:43:49.121 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-09-17T11:43:49.121 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=0 cursor=12:6cac518f:::0:head 2024-09-17T11:43:49.121 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=15 cursor=12:863748b0:::15:head 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:863748b0:::15:head 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=11 cursor=12:89d3ae78:::11:head 2024-09-17T11:43:49.122 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=5 cursor=12:b29083e3:::5:head 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=8 cursor=12:bd63b0f1:::8:head 2024-09-17T11:43:49.123 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=6 cursor=12:c4fdafeb:::6:head 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=3 cursor=12:cfc208b3:::3:head 2024-09-17T11:43:49.124 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=4 cursor=12:d83876eb:::4:head 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=12 cursor=12:de5d7c5f:::12:head 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-09-17T11:43:49.125 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=9 cursor=12:e960b815:::9:head 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:e960b815:::9:head 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : oid=2 cursor=12:f905c69b:::2:head 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-09-17T11:43:49.126 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-09-17T11:43:49.127 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:f905c69b:::2:head expected=12:f905c69b:::2:head 2024-09-17T11:43:49.127 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-09-17T11:43:49.127 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=2 expected=2 2024-09-17T11:43:49.127 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-09-17T11:43:49.127 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:c4fdafeb:::6:head expected=12:c4fdafeb:::6:head 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=6 expected=6 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:bd63b0f1:::8:head expected=12:bd63b0f1:::8:head 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=8 expected=8 2024-09-17T11:43:49.128 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:e960b815:::9:head 2024-09-17T11:43:49.129 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:e960b815:::9:head expected=12:e960b815:::9:head 2024-09-17T11:43:49.129 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-09-17T11:43:49.129 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=9 expected=9 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:de5d7c5f:::12:head expected=12:de5d7c5f:::12:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=12 expected=12 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:d83876eb:::4:head expected=12:d83876eb:::4:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=4 expected=4 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:cfc208b3:::3:head expected=12:cfc208b3:::3:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=3 expected=3 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-09-17T11:43:49.130 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:b29083e3:::5:head expected=12:b29083e3:::5:head 2024-09-17T11:43:49.131 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: pgmap v47: 580 pgs: 448 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: Health check failed: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3030815591' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2925658233' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/105546453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4164395612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/194336557' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: pgmap v47: 580 pgs: 448 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: Health check failed: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.905 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3030815591' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2925658233' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/105546453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4164395612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.906 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.907 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.907 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.907 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/194336557' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.907 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.907 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: pgmap v47: 580 pgs: 448 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: Health check failed: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi049-75787-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi049-75736-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi049-75753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi049-76109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi049-75918-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi049-75804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi049-75865-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi049-76174-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi049-76013-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi049-76193-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/888971847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi049-75722-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi049-76764-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi049-76874-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:49.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3030815591' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2925658233' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/105546453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4164395612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/194336557' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:49.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:50.370 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=5 expecte list_parallel: process_1_[76599]: starting. 2024-09-17T11:43:50.370 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: creating pool ceph_test_rados_list_parallel.smithi049-76549 2024-09-17T11:43:50.370 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: created object 0... 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: created object 25... 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: created object 49... 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: finishing. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_1_[76599]: shutting down. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: starting. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: listing objects. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: listed object 0... 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: listed object 25... 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: saw 50 objects 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_2_[76601]: shutting down. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: starting. 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: creating pool ceph_test_rados_list_parallel.smithi049-76549 2024-09-17T11:43:50.371 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: created object 0... 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: created object 25... 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: created object 49... 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: finishing. 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_3_[77430]: shutting down. 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: starting. 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: listing objects. 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: listed object 0... 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: listed object 25... 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: saw 42 objects 2024-09-17T11:43:50.372 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_4_[77431]: shutting down. 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: starting. 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: removed 25 objects... 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: removed half of the objects 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: removed 50 objects... 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: removed 50 objects 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_5_[77432]: shutting down. 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: starting. 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: creating pool ceph_test_rados_list_parallel.smithi049-76549 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: created object 0... 2024-09-17T11:43:50.373 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: created object 25... 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: created object 49... 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: finishing. 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_6_[77478]: shutting down. 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: starting. 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: listing objects. 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: listed object 0... 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: listed object 25... 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: listed object 50... 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: saw 56 objects 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_7_[77479]: shutting down. 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.374 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: starting. 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: added 25 objects... 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: added half of the objects 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: added 50 objects... 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: added 50 objects 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_8_[77480]: shutting down. 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.375 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.625 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ********************** api_cmd_pp: Running main() from gmock_main.cc 2024-09-17T11:43:50.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-17T11:43:50.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-17T11:43:50.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-17T11:43:50.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-17T11:43:50.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (156 ms) 2024-09-17T11:43:50.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-17T11:43:50.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (82 ms) 2024-09-17T11:43:50.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-17T11:43:50.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2827 ms) 2024-09-17T11:43:50.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (3065 ms total) 2024-09-17T11:43:50.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: 2024-09-17T11:43:50.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-17T11:43:50.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (3065 ms total) 2024-09-17T11:43:50.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-17T11:43:50.628 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: Running main() from gmock_main.cc 2024-09-17T11:43:50.628 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [----------] Global test environment set-up. 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2885 ms) 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [----------] 1 test from neocls_handler_error (2885 ms total) 2024-09-17T11:43:50.629 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: 2024-09-17T11:43:50.630 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [----------] Global test environment tear-down 2024-09-17T11:43:50.630 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2885 ms total) 2024-09-17T11:43:50.630 INFO:tasks.workunit.client.0.smithi049.stdout: handler_error: [ PASSED ] 1 test. 2024-09-17T11:43:50.632 INFO:tasks.workunit.client.0.smithi049.stdout: cls: Running main() from gmock_main.cc 2024-09-17T11:43:50.632 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [==========] Running 1 test from 1 test suite. 2024-09-17T11:43:50.633 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [----------] Global test environment set-up. 2024-09-17T11:43:50.633 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [----------] 1 test from NeoRadosCls 2024-09-17T11:43:50.633 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-09-17T11:43:50.633 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [ OK ] NeoRadosCls.DNE (2977 ms) 2024-09-17T11:43:50.633 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [----------] 1 test from NeoRadosCls (2977 ms total) 2024-09-17T11:43:50.634 INFO:tasks.workunit.client.0.smithi049.stdout: cls: 2024-09-17T11:43:50.634 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [----------] Global test environment tear-down 2024-09-17T11:43:50.634 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [==========] 1 test from 1 test suite ran. (2977 ms total) 2024-09-17T11:43:50.634 INFO:tasks.workunit.client.0.smithi049.stdout: cls: [ PASSED ] 1 test. 2024-09-17T11:43:50.635 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-17T11:43:50.635 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-09-17T11:43:50.635 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-17T11:43:50.635 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-09-17T11:43:50.636 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-17T11:43:50.636 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-17T11:43:50.636 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-17T11:43:50.636 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (306 ms) 2024-09-17T11:43:50.636 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-17T11:43:50.637 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (4 ms) 2024-09-17T11:43:50.637 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-17T11:43:50.637 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (11 ms) 2024-09-17T11:43:50.637 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-17T11:43:50.637 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (21 ms) 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (5 ms) 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-17T11:43:50.638 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (4 ms) 2024-09-17T11:43:50.639 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-17T11:43:50.639 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (7 ms) 2024-09-17T11:43:50.639 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-17T11:43:50.639 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-09-17T11:43:50.639 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (4 ms) 2024-09-17T11:43:50.640 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-17T11:43:50.641 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-17T11:43:50.641 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-17T11:43:50.641 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (9 ms) 2024-09-17T11:43:50.641 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-17T11:43:50.641 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (7 ms) 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (3 ms) 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (414 ms total) 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-17T11:43:50.642 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2997 ms total) 2024-09-17T11:43:50.643 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (104 ms) 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (210 ms) 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2823 ms) 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-17T11:43:50.729 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583142+0000 mon.a [INF] from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583219+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583281+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583360+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583478+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.583526+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.644952+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.695265+0000 mon.c [INF] from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.699347+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.707304+0000 mon.c [INF] from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.713263+0000 mon.c [INF] from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.733636+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.730 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.744236+0000 mon.c [INF] from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.744439+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.750149+0000 mon.c [INF] from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.754587+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.756953+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.757902+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.766655+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.771203+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.771330+0000 mon.c [INF] from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:49.772061+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.599503+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:50.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.599613+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:50.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.599733+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:50.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.599869+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.599991+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.600100+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIo********* 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: starting. 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: creating pool ceph_test_rados_list_parallel.smithi049-76549 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: created object 0... 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: created object 25... 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: created object 49... 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: finishing. 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_9_[77526]: shutting down. 2024-09-17T11:43:50.872 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: starting. 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: listing objects. 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: listed object 0... 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: listed object 25... 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: listed object 50... 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: listed object 75... 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: saw 96 objects 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_10_[77527]: shutting down. 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.873 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: starting. 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: removed 25 objects... 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: removed half of the objects 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: removed 50 objects... 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: removed 50 objects 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_13_[77530]: shutting down. 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: starting. 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: added 25 objects... 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: added half of the objects 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: added 50 objects... 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: added 50 objects 2024-09-17T11:43:50.874 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_11_[77528]: shutting down. 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: starting. 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: added 25 objects... 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: added half of the objects 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: added 50 objects... 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: added 50 objects 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_12_[77529]: shutting down. 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.875 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: starting. 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: creating pool ceph_test_rados_list_parallel.smithi049-76549 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: created object 0... 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: created object 25... 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: created object 49... 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: finishing. 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_14_[77606]: shutting down. 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: starting. 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listing objects. 2024-09-17T11:43:50.876 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 0... 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T11:43:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:50.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T11:43:50.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1887957199' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:50.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2839790615' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:50.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:51.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T11:43:51.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1887957199' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3858102541' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi049-76082-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi049-76561-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76231-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76428-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi049-76181-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2839790615' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi049-76288-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi049-76764-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi049-76874-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi049-75753-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.157 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.158 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T11:43:51.158 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1887957199' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.158 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2839790615' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.158 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 25... 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 50... 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 75... 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 100... 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: listed object 125... 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: saw 150 objects 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_15_[77607]: shutting down. 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.639 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: starting. 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: added 25 objects... 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: added half of the objects 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: added 50 objects... 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: added 50 objects 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: process_16_[77608]: shutting down. 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******************************* 2024-09-17T11:43:51.640 INFO:tasks.workunit.client.0.smithi049.stdout: list_parallel: ******* SUCCESS ********** 2024-09-17T11:43:51.707 INFO:tasks.workunit.client.0.smithi049.stdout:ECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.707 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.600214+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi049-76174-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.605018+0000 mon.c [INF] from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.605190+0000 mon.c [INF] from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.611702+0000 mon.b [INF] from='client.? 172.21.15.49:0/1887957199' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.612024+0000 mon.b [INF] from='client.? 172.21.15.49:0/2839790615' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.613528+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.621976+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.623424+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.626087+0000 mon.a [INF] from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.626417+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.626746+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.723709+0000 mon.c [INF] from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:51.708 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.724074+0000 client.admin [INF] onexx 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.728495+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.739978+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.766052+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:50.774862+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.409242+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.409839+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.410278+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.410755+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.411184+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.411635+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.412073+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.412532+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:43:51.709 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.412966+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.413389+0000 mon.c [INF] from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.604017+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.604078+0000 mon.a [INF] from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.604125+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.604189+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:51.710 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.632052+0000 mon.c [INF] from='client.? 172.21.15.49:0/2337192434' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: pgmap v50: 964 pgs: 832 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: onexx 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:43:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: pgmap v50: 964 pgs: 832 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:52.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: onexx 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:52.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:52.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2337192434' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2337192434' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: pgmap v50: 964 pgs: 832 unknown, 132 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: onexx 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi049-75722-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2337192434' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:52.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:43:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:43:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.4 deep-scrub starts 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.4 deep-scrub ok 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: twoxx 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.6 deep-scrub starts 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.4 deep-scrub starts 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.4 deep-scrub ok 2024-09-17T11:43:53.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: twoxx 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.6 deep-scrub starts 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.6 deep-scrub ok 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.3 deep-scrub starts 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.3 deep-scrub ok 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.0 deep-scrub starts 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.0 deep-scrub ok 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.1 deep-scrub starts 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: 15.1 deep-scrub ok 2024-09-17T11:43:53.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.6 deep-scrub ok 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.3 deep-scrub starts 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.3 deep-scrub ok 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.0 deep-scrub starts 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.0 deep-scrub ok 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.1 deep-scrub starts 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: 15.1 deep-scrub ok 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:53.157 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.158 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T11:43:53.159 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.160 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:43:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.4 deep-scrub starts 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.4 deep-scrub ok 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: twoxx 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.6 deep-scrub starts 2024-09-17T11:43:53.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.6 deep-scrub ok 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.3 deep-scrub starts 2024-09-17T11:43:53.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.3 deep-scrub ok 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.0 deep-scrub starts 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.0 deep-scrub ok 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.1 deep-scrub starts 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: 15.1 deep-scrub ok 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi049-75753-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi049-76174-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi049-75865-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2324031554' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:53.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:53.182 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:53.660 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.638109+0000 mon.b [INF] from='client.? 172.21.15.49:0/2324031554' entit open_pools_parallel: process_1_[76683]: starting. 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: creating pool ceph_test_rados_open_pools_parallel.smithi049-76606 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: created object 0... 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: created object 25... 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: created object 49... 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: finishing. 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_1_[76683]: shutting down. 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_2_[76684]: starting. 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_2_[76684]: rados_pool_create. 2024-09-17T11:43:53.661 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_2_[76684]: rados_ioctx_create. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_2_[76684]: shutting down. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: ******************************* 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: starting. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: creating pool ceph_test_rados_open_pools_parallel.smithi049-76606 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: created object 0... 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: created object 25... 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: created object 49... 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: finishing. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_3_[77662]: shutting down. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: ******************************* 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_4_[77663]: starting. 2024-09-17T11:43:53.662 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_4_[77663]: rados_pool_create. 2024-09-17T11:43:53.663 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_4_[77663]: rados_ioctx_create. 2024-09-17T11:43:53.663 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: process_4_[77663]: shutting down. 2024-09-17T11:43:53.663 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: ******************************* 2024-09-17T11:43:53.663 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: ******************************* 2024-09-17T11:43:53.663 INFO:tasks.workunit.client.0.smithi049.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-17T11:43:53.670 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: starting. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: creating pool ceph_test_rados_delete_pools_parallel.smithi049-76600 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: created object 0... 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: created object 25... 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: created object 49... 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: finishing. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_1_[76640]: shutting down. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_2_[76642]: starting. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_2_[76642]: deleting pool ceph_test_rados_delete_pools_parallel.smithi049-76600 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_2_[76642]: shutting down. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******************************* 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: starting. 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: creating pool ceph_test_rados_delete_pools_parallel.smithi049-76600 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: created object 0... 2024-09-17T11:43:53.671 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: created object 25... 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: created object 49... 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: finishing. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_3_[77654]: shutting down. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******************************* 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: starting. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: listing objects. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: listed object 0... 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: listed object 25... 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: saw 50 objects 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_5_[77656]: shutting down. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******************************* 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_4_[77655]: starting. 2024-09-17T11:43:53.672 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_4_[77655]: deleting pool ceph_test_rados_delete_pools_parallel.smithi049-76600 2024-09-17T11:43:53.673 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: process_4_[77655]: shutting down. 2024-09-17T11:43:53.673 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******************************* 2024-09-17T11:43:53.673 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******************************* 2024-09-17T11:43:53.673 INFO:tasks.workunit.client.0.smithi049.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-17T11:43:53.817 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: seed 75787 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (222 ms) 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-09-17T11:43:53.818 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (3 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (253 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (1 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (1 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (3 ms) 2024-09-17T11:43:53.819 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-09-17T11:43:53.820 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (8 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (3 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (2 ms) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (538 ms total) 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: 2024-09-17T11:43:53.821 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1142 ms) 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (24 ms) 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-17T11:43:53.822 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (3 ms) 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: pgmap v53: 820 pgs: 288 creating+peering, 532 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 549 op/s 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.7 deep-scrub starts 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.7 deep-scrub ok 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1052355048' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.5 deep-scrub starts 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.5 deep-scrub ok 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.2 deep-scrub starts 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: 15.2 deep-scrub ok 2024-09-17T11:43:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3278585923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1224049218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:54.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: pgmap v53: 820 pgs: 288 creating+peering, 532 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 549 op/s 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.7 deep-scrub starts 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.7 deep-scrub ok 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1052355048' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.5 deep-scrub starts 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.5 deep-scrub ok 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.2 deep-scrub starts 2024-09-17T11:43:54.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: 15.2 deep-scrub ok 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3278585923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1224049218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.155 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T11:43:54.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.156 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: pgmap v53: 820 pgs: 288 creating+peering, 532 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 549 op/s 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.7 deep-scrub starts 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.7 deep-scrub ok 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1052355048' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.5 deep-scrub starts 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.5 deep-scrub ok 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.2 deep-scrub starts 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: 15.2 deep-scrub ok 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:54.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: osdmap e72: 8 total, 8 up, 8 in 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3278585923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1224049218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T11:43:54.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.268 INFO:tasks.workunit.client.0.smithi049.stdout: ap cmd: Running main() from gmock_main.cc 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [----------] Global test environment set-up. 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1849 ms) 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2026 ms) 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-09-17T11:43:54.269 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (2622 ms) 2024-09-17T11:43:54.270 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6497 ms total) 2024-09-17T11:43:54.270 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: 2024-09-17T11:43:54.270 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [----------] Global test environment tear-down 2024-09-17T11:43:54.270 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6497 ms total) 2024-09-17T11:43:54.270 INFO:tasks.workunit.client.0.smithi049.stdout: cmd: [ PASSED ] 3 tests. 2024-09-17T11:43:54.740 INFO:tasks.workunit.client.0.smithi049.stdout:y='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:54.740 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.645333+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:54.740 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.646480+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.650516+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi049-76764-1"} : dispatch 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:51.651296+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:52.725379+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.613768+0000 mon.a [INF] from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi049-76181-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.613865+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi049-76764-1"}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.613948+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.613995+0000 mon.a [INF] from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi049-75736-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.614039+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.614109+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.620289+0000 mon.b [INF] from='client.? 172.21.15.49:0/4235434111' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.620516+0000 mon.b [INF] from='client.? 172.21.15.49:0/1716238953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.741 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.651904+0000 mon.b [INF] from='client.? 172.21.15.49:0/3278585923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.742 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.652201+0000 mon.b [INF] from='client.? 172.21.15.49:0/1224049218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.654967+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.655416+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.655665+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.656111+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.656336+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.664728+0000 mon.a [INF] from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.678908+0000 mon.b [INF] from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.679847+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:53.857155+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.236188+0000 mon.a [WRN] Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.241909+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:43:54.755 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.241996+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242068+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242126+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242168+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242226+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242290+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242332+0000 mon.a [INF] from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242367+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.242428+0000 mon.a [INF] from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.248538+0000 mon.b [INF] from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.252434+0000 mon.a [INF] from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"} : dispatch 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.253120+0000 mon.c [INF] from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.253489+0000 mon.c [INF] from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:54.756 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.264818+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.265347+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.265666+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.266065+0000 mon.a [INF] from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.284760+0000 mon.c [INF] from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.285242+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.739947+0000 mon.c [INF] from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:54.757 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.740075+0000 client.admin [INF] threexx 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: 15.9 deep-scrub starts 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: 15.9 deep-scrub ok 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: 15.8 deep-scrub starts 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: 15.8 deep-scrub ok 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:43:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"} : dispatch 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: threexx 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-17T11:43:55.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: 15.9 deep-scrub starts 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: 15.9 deep-scrub ok 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: 15.8 deep-scrub starts 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: 15.8 deep-scrub ok 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-17T11:43:55.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.155 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:55.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: threexx 2024-09-17T11:43:55.156 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: 15.9 deep-scrub starts 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: 15.9 deep-scrub ok 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: 15.8 deep-scrub starts 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: 15.8 deep-scrub ok 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi049-75865-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi049-75918-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi049-76874-1"}]': finished 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-4", "overlaypool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi049-76288-1"}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi049-75722-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1467091206' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi049-75736-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi049-76764-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi049-75787-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: osdmap e73: 8 total, 8 up, 8 in 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:55.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:55.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:55.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:55.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: threexx 2024-09-17T11:43:55.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: pgmap v57: 900 pgs: 416 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 568 op/s 2024-09-17T11:43:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-17T11:43:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: fourxx 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3970153093' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"}]': finished 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: pgmap v57: 900 pgs: 416 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 568 op/s 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: fourxx 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3970153093' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"}]': finished 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T11:43:56.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:56.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"} : dispatch 2024-09-17T11:43:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: pgmap v57: 900 pgs: 416 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 39 MiB/s wr, 568 op/s 2024-09-17T11:43:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-17T11:43:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/599371636' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: fourxx 2024-09-17T11:43:56.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-17T11:43:56.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3970153093' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:56.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:56.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:43:56.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-4-cache", "mode": "writeback"}]': finished 2024-09-17T11:43:56.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:56.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:56.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/968981445' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:56.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi049-76874-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:43:56.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1228608447' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4004241128' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: osdmap e74: 8 total, 8 up, 8 in 2024-09-17T11:43:56.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"} : dispatch 2024-09-17T11:43:56.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"} : dispatch 2024-09-17T11:43:56.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:56.182 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.182 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:43:56.182 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"} : dispatch 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: Running main() from gmock_main.cc 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] Global test environment set-up. 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (223 ms) 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-17T11:43:56.266 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: no timeout :/ 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: no timeout :/ 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: no timeout :/ 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: no timeout :/ 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: no timeout :/ 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (42 ms) 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (5 ms) 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-17T11:43:56.267 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (3 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (3 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (3 ms) 2024-09-17T11:43:56.268 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.RmXattr (9 ms) 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] 14 tests from LibRadosIo (307 ms total) 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1169 ms) 2024-09-17T11:43:56.269 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (9 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (8 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-09-17T11:43:56.270 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1220 ms total) 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [----------] Global test environment tear-down 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [==========] 24 tests from 2 test suites ran. (8991 ms total) 2024-09-17T11:43:56.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_io: [ PASSED ] 24 tests. 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: Running main() from gmock_main.cc 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] Global test environment set-up. 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStat.Stat (217 ms) 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (4 ms) 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStat.StatNS (13 ms) 2024-09-17T11:43:56.272 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] 5 tests from LibRadosStat (237 ms total) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1172 ms) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (13 ms) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-17T11:43:56.273 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1 ms) 2024-09-17T11:43:56.274 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1187 ms total) 2024-09-17T11:43:56.274 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: 2024-09-17T11:43:56.274 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [----------] Global test environment tear-down 2024-09-17T11:43:56.274 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (8812 ms total) 2024-09-17T11:43:56.274 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat: [ PASSED ] 9 tests. 2024-09-17T11:43:56.285 INFO:tasks.workunit.client.0.smithi049.stdout: list: Running main() from gmock_main.cc 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [----------] Global test environment set-up. 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [----------] 3 tests from NeoradosList 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ OK ] NeoradosList.ListObjects (2782 ms) 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3044 ms) 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ OK ] NeoradosList.ListObjectsMany (2619 ms) 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [----------] 3 tests from NeoradosList (8445 ms total) 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [----------] Global test environment tear-down 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [==========] 3 tests from 1 test suite ran. (8446 ms total) 2024-09-17T11:43:56.286 INFO:tasks.workunit.client.0.smithi049.stdout: list: [ PASSED ] 3 tests. 2024-09-17T11:43:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:43:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:43:56.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-17T11:43:56.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:56.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:43:56.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"}]': finished 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3271566763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4} : dispatch 2024-09-17T11:43:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"}]': finished 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3271566763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4} : dispatch 2024-09-17T11:43:57.010 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [----------] Global test environment set-up. 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5107 ms) 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ OK ] LibRadosService.RegisterLate (90 ms) 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: cluster: 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: id: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: health: HEALTH_WARN 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 13 pool(s) do not have an application enabled 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.011 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: services: 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mgr: y(active, since 86s), standbys: x 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: data: 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pools: 29 pools, 820 pgs 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: objects: 263 objects, 145 MiB 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: usage: 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pgs: 35.122% pgs not active 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 532 active+clean 2024-09-17T11:43:57.012 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 288 creating+peering 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: io: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: client: 12 MiB/s rd, 37 MiB/s wr, 256 op/s rd, 292 op/s wr 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: cluster: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: id: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: health: HEALTH_WARN 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 13 pool(s) do not have an application enabled 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: services: 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-17T11:43:57.013 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mgr: y(active, since 88s), standbys: x 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: data: 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pools: 29 pools, 820 pgs 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: objects: 263 objects, 145 MiB 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: usage: 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pgs: 35.122% pgs not active 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 532 active+clean 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 288 creating+peering 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: io: 2024-09-17T11:43:57.014 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: client: 12 MiB/s rd, 37 MiB/s wr, 256 op/s rd, 292 op/s wr 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: cluster: 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: id: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: health: HEALTH_WARN 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 1 cache pools are missing hit_sets 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 15 pool(s) do not have an application enabled 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: services: 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: mgr: y(active, since 90s), standbys: x 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-17T11:43:57.015 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-17T11:43:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi049-76764-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi049-75753-16"}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi049-76174-7"}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-4"}]': finished 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: osdmap e75: 8 total, 8 up, 8 in 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3271566763' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"} : dispatch 2024-09-17T11:43:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"} : dispatch 2024-09-17T11:43:57.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:57.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:57.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4} : dispatch 2024-09-17T11:43:57.271 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zo watch_notify: Running main() from gmock_main.cc 2024-09-17T11:43:57.271 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [----------] Global test environment set-up. 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: handle_notify cookie 94077744298016 notify_id 292057776128 notifier_gid 24782 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2713 ms) 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: Trying... 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: handle_notify cookie 94077757038128 notify_id 304942678017 notifier_gid 24929 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: Waiting for 3.000000000s 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: Timed out. 2024-09-17T11:43:57.272 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: Flushing... 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: Flushed... 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (6654 ms) 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9367 ms total) 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [----------] Global test environment tear-down 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9367 ms total) 2024-09-17T11:43:57.273 INFO:tasks.workunit.client.0.smithi049.stdout: watch_notify: [ PASSED ] 2 tests. 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: seed 76181 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (253 ms) 2024-09-17T11:43:57.279 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (52 ms) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (3 ms) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (14 ms) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (323 ms total) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (709 ms) 2024-09-17T11:43:57.280 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (6 ms) 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (717 ms total) 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9823 ms total) 2024-09-17T11:43:57.281 INFO:tasks.workunit.client.0.smithi049.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-17T11:43:57.761 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: got: 2024-09-17T11:43:54.740250+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7134 ms) 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (10271 ms total) 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (10271 ms total) 2024-09-17T11:43:57.762 INFO:tasks.workunit.client.0.smithi049.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-17T11:43:57.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:43:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: pgmap v60: 772 pgs: 288 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:43:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2073168835' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4}]': finished 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1500637146' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: pgmap v60: 772 pgs: 288 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2073168835' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4}]': finished 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1500637146' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:58.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:58.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: pgmap v60: 772 pgs: 288 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:43:58.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:43:58.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2073168835' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi049-76874-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/910296585' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi049-76181-7"}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4166473365' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-4", "tierpool": "test-rados-api-smithi049-76013-4-cache"}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2194320385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi049-75722-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76288-6", "pg_num": 4}]': finished 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: osdmap e76: 8 total, 8 up, 8 in 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1500637146' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:43:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:58.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:59.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: osdmap e77: 8 total, 8 up, 8 in 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:43:59.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:43:59.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:00.284 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: Running main() from gmock_main.cc 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [----------] Global test environment set-up. 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5877 ms) 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6660 ms) 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12537 ms total) 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [----------] Global test environment tear-down 2024-09-17T11:44:00.285 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12537 ms total) 2024-09-17T11:44:00.286 INFO:tasks.workunit.client.0.smithi049.stdout: ec_io: [ PASSED ] 2 tests. 2024-09-17T11:44:00.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: seed 75918 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (208 ms) 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (4 ms) 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1080 ms) 2024-09-17T11:44:00.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1041 ms) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (9 ms) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (13 ms) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2363 ms total) 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: 2024-09-17T11:44:00.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1121 ms) 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1060 ms) 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1015 ms) 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-09-17T11:44:00.294 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (2 ms) 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3218 ms total) 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-17T11:44:00.295 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (12938 ms total) 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: Running main() from gmock_main.cc 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] Global test environment set-up. 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-17T11:44:00.296 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (221 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.LockShared (3 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1107 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1019 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (6 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-09-17T11:44:00.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-17T11:44:00.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (10 ms) 2024-09-17T11:44:00.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-17T11:44:00.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (6 ms) 2024-09-17T11:44:00.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] 8 tests from LibRadosLock (2376 ms total) 2024-09-17T11:44:00.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1123 ms) 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (7 ms) 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1064 ms) 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-09-17T11:44:00.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5 ms) 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3216 ms total) 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [----------] Global test environment tear-down 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (12951 ms total) 2024-09-17T11:44:00.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-17T11:44:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: pgmap v63: 688 pgs: 128 unknown, 140 creating+peering, 420 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:44:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2731300902' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"} : dispatch 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: pgmap v63: 688 pgs: 128 unknown, 140 creating+peering, 420 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2731300902' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911422452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1095276034' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911422452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1095276034' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:43:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: pgmap v63: 688 pgs: 128 unknown, 140 creating+peering, 420 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "overlaypool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2944964367' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: osdmap e78: 8 total, 8 up, 8 in 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3260993621' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2763361259' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2145805949' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2731300902' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"} : dispatch 2024-09-17T11:44:00.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911422452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1095276034' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:00.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:43:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: Running main() from gmock_main.cc 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [----------] Global test environment set-up. 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolList (1760 ms) 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (2043 ms) 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-17T11:44:01.330 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (1995 ms) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (3619 ms) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2012 ms) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2041 ms) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [----------] 6 tests from NeoRadosPools (13470 ms total) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [----------] Global test environment tear-down 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [==========] 6 tests from 1 test suite ran. (13471 ms total) 2024-09-17T11:44:01.331 INFO:tasks.workunit.client.0.smithi049.stdout: pool: [ PASSED ] 6 tests. 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"}]': finished 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:01.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2"} : dispatch 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1054919235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2"} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1054919235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76288-6", "mode": "writeback"}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi049-76764-2"}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi049-75865-10"}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi049-75918-10"}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi049-76082-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi049-75722-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:01.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3020244432' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2"} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: osdmap e79: 8 total, 8 up, 8 in 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1054919235' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"} : dispatch 2024-09-17T11:44:01.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: pgmap v66: 680 pgs: 320 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-17T11:44:02.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T11:44:02.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:02.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: pgmap v66: 680 pgs: 320 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-17T11:44:02.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:02.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:02.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:02.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-17T11:44:02.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T11:44:02.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:02.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: pgmap v66: 680 pgs: 320 unknown, 4 creating+peering, 356 active+clean; 145 MiB data, 758 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-17T11:44:02.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:02.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi049-76193-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:02.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi049-75987-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi049-76874-2"}]': finished 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: osdmap e80: 8 total, 8 up, 8 in 2024-09-17T11:44:02.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:02.431 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.431 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:02.431 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3147968375' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1000399788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3147968375' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1000399788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:03.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi049-75787-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:03.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi049-76874-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: osdmap e81: 8 total, 8 up, 8 in 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3147968375' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1000399788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1149 ms) 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-17T11:44:03.433 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (5 ms) 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1154 ms total) 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (100 ms) 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4007 ms) 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-17T11:44:03.434 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (4 ms) 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350034000 notify_id 339302416390 notifier_gid 15207 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350053056 notify_id 339302416384 notifier_gid 15207 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350053056 notify_id 339302416391 notifier_gid 15207 2024-09-17T11:44:03.435 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350053056 notify_id 339302416392 notifier_gid 15207 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350053056 notify_id 339302416393 notifier_gid 15207 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-17T11:44:03.436 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350076768 notify_id 339302416386 notifier_gid 15207 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: trying... 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350076768 notify_id 339302416387 notifier_gid 15207 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: timed out 2024-09-17T11:44:03.437 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: flushing 2024-09-17T11:44:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: pgmap v69: 600 pgs: 136 creating+peering, 464 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2024-09-17T11:44:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4213314189' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: pgmap v69: 600 pgs: 136 creating+peering, 464 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4213314189' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:04.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: pgmap v69: 600 pgs: 136 creating+peering, 464 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 6 op/s 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi049-75722-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi049-76109-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2065565712' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2557283926' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: osdmap e82: 8 total, 8 up, 8 in 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4213314189' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:04.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.324 INFO:tasks.workunit.client.0.smithi049.stdout: api_watc api_watch_notify: Running main() from gmock_main.cc 2024-09-17T11:44:05.324 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify_test_cb 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (209 ms) 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-17T11:44:05.325 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93901020167712 err -107 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (9 ms) 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93901020179760 err -107 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-17T11:44:05.326 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (9 ms) 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808833 cookie 93901020197312 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (6 ms) 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808833 cookie 93901020197312 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (6 ms) 2024-09-17T11:44:05.327 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-17T11:44:05.328 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808832 cookie 93901020197312 2024-09-17T11:44:05.328 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-09-17T11:44:05.328 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-17T11:44:05.328 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808834 cookie 93901020235488 2024-09-17T11:44:05.328 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808834 cookie 93901020237792 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (8 ms) 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 287762808835 cookie 93901020235488 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 292057776132 cookie 93901020235488 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-09-17T11:44:05.329 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-17T11:44:05.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-17T11:44:05.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93901020235488 err -107 2024-09-17T11:44:05.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_cb from 24727 notify_id 322122547203 cookie 93901020235488 2024-09-17T11:44:05.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-09-17T11:44:05.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93901020235488 err -107 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9269 ms total) 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: 2024-09-17T11:44:05.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-17T11:44:05.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-17T11:44:05.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: watch_notify_test_cb 2024-09-17T11:44:05.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1101 ms) 2024-09-17T11:44:05.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1101 ms total) 2024-09-17T11:44:05.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: 2024-09-17T11:44:05.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-17T11:44:05.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (17864 ms total) 2024-09-17T11:44:05.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: pool 'PoolQuotaPP_smithi049-75736-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:05.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: pool 'PoolQuotaPP_smithi049-75736-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.406 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:05.407 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: pool 'PoolQuotaPP_smithi049-75736-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi049-76874-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi049-75987-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: osdmap e83: 8 total, 8 up, 8 in 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1764173616' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"} : dispatch 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:06.312 INFO:tasks.workunit.client.0.smithi049.stdout:i_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-17T11:44:06.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-09-17T11:44:06.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-17T11:44:06.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (4 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-09-17T11:44:06.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (8462 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (1079 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-09-17T11:44:06.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (10761 ms total) 2024-09-17T11:44:06.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: 2024-09-17T11:44:06.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-17T11:44:06.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (19008 ms total) 2024-09-17T11:44:06.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_io_pp: [ PASSED ] 40 tests. 2024-09-17T11:44:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: pgmap v72: 688 pgs: 200 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1690002913' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: pgmap v72: 688 pgs: 200 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1690002913' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: pgmap v72: 688 pgs: 200 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-09-17T11:44:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi049-76193-12"}]': finished 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: osdmap e84: 8 total, 8 up, 8 in 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"} : dispatch 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1690002913' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4167251374' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T11:44:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4167251374' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/332835488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi049-75787-24"}]': finished 2024-09-17T11:44:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi049-75722-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:07.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: osdmap e85: 8 total, 8 up, 8 in 2024-09-17T11:44:07.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4167251374' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:07.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:07.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: pgmap v75: 656 pgs: 200 unknown, 32 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2039776717' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:08.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: pgmap v75: 656 pgs: 200 unknown, 32 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2039776717' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:08.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:08.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: pgmap v75: 656 pgs: 200 unknown, 32 creating+peering, 424 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:08.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:08.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76458-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: osdmap e86: 8 total, 8 up, 8 in 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2039776717' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"} : dispatch 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: Running main() from gmock_main.cc 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [----------] Global test environment set-up. 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6445 ms) 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-09-17T11:44:09.324 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (7038 ms) 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8034 ms) 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [----------] 3 tests from NeoradosECList (21517 ms total) 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [----------] Global test environment tear-down 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (21518 ms total) 2024-09-17T11:44:09.325 INFO:tasks.workunit.client.0.smithi049.stdout: ec_list: [ PASSED ] 3 tests. 2024-09-17T11:44:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688074209' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688074209' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-7"}]': finished 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: osdmap e87: 8 total, 8 up, 8 in 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"} : dispatch 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688074209' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: pgmap v78: 680 pgs: 96 creating+peering, 1 creating+activating, 32 unknown, 551 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-17T11:44:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[30017]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: pgmap v78: 680 pgs: 96 creating+peering, 1 creating+activating, 32 unknown, 551 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:10.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:09 smithi049 ceph-mon[40223]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T11:44:10.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: pgmap v78: 680 pgs: 96 creating+peering, 1 creating+activating, 32 unknown, 551 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-17T11:44:10.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:10.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:10.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742845137' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi049-76874-3"}]': finished 2024-09-17T11:44:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi049-75722-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:09 smithi129 ceph-mon[33364]: osdmap e88: 8 total, 8 up, 8 in 2024-09-17T11:44:11.330 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: Running main() from gmock_main.cc 2024-09-17T11:44:11.330 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-09-17T11:44:11.330 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [----------] Global test environment set-up. 2024-09-17T11:44:11.330 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2731 ms) 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3016 ms) 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (2624 ms) 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3003 ms) 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3019 ms) 2024-09-17T11:44:11.331 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (2982 ms) 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6037 ms) 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (23412 ms total) 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [----------] Global test environment tear-down 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (23412 ms total) 2024-09-17T11:44:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: write_operations: [ PASSED ] 7 tests. 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-17T11:44:11.350 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (3054 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3039 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2624 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3005 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3039 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2974 ms) 2024-09-17T11:44:11.351 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (6043 ms) 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (23779 ms total) 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (23779 ms total) 2024-09-17T11:44:11.352 INFO:tasks.workunit.client.0.smithi049.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-17T11:44:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[30017]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:44:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[30017]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T11:44:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:11.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[40223]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:44:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[40223]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T11:44:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:11.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:10 smithi129 ceph-mon[33364]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:44:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:10 smithi129 ceph-mon[33364]: osdmap e89: 8 total, 8 up, 8 in 2024-09-17T11:44:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: pgmap v81: 680 pgs: 64 creating+peering, 1 creating+activating, 128 unknown, 487 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-17T11:44:12.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:12.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:12.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T11:44:12.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2478725934' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:12.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: pgmap v81: 680 pgs: 64 creating+peering, 1 creating+activating, 128 unknown, 487 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-17T11:44:12.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:12.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:12.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T11:44:12.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2478725934' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:11 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: pgmap v81: 680 pgs: 64 creating+peering, 1 creating+activating, 128 unknown, 487 active+clean; 145 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-17T11:44:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: osdmap e90: 8 total, 8 up, 8 in 2024-09-17T11:44:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2478725934' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:11 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:12.926 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=infra.usagestats t=2024-09-17T11:44:12.453927693Z level=info msg="Usage stats are ready to report" 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2785154025' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"} : dispatch 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:13.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2785154025' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"} : dispatch 2024-09-17T11:44:13.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:13.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi049-75722-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:13.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:13.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: osdmap e91: 8 total, 8 up, 8 in 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2785154025' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"} : dispatch 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: pgmap v84: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"}]': finished 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:44:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:14.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: pgmap v84: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"}]': finished 2024-09-17T11:44:14.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T11:44:14.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:44:14.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:14.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: pgmap v84: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1796912038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4"}]': finished 2024-09-17T11:44:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: osdmap e92: 8 total, 8 up, 8 in 2024-09-17T11:44:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"} : dispatch 2024-09-17T11:44:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"} : dispatch 2024-09-17T11:44:15.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"}]': finished 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3415405847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"}]': finished 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T11:44:15.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3415405847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:15.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi049-76288-4", "tierpool": "test-rados-api-smithi049-76288-6"}]': finished 2024-09-17T11:44:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-9", "mode": "writeback"}]': finished 2024-09-17T11:44:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: osdmap e93: 8 total, 8 up, 8 in 2024-09-17T11:44:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3415405847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:15.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:16.302 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: pgmap v87: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:44:16.302 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:16.302 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: pgmap v87: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.304 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T11:44:16.305 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.305 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.305 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.305 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:16.334 INFO:tasks.workunit.client.0.smithi049.stdout:h_notify_pp: flushed 2024-09-17T11:44:16.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3003 ms) 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: trying... 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350076768 notify_id 352187318273 notifier_gid 15207 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: timed out 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: flushing 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: flushed 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: List watches 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify2 2024-09-17T11:44:16.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350076768 notify_id 365072220162 notifier_gid 15207 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify2 done 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: watch_check 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: unwatch2 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: flushing 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: done 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3052 ms) 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: List watches 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify2 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: handle_notify cookie 94299350076768 notify_id 377957122051 notifier_gid 15207 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: notify2 done 2024-09-17T11:44:16.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: watch_check 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: unwatch2 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: flushing 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: done 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3013 ms) 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16216 ms total) 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28828 ms total) 2024-09-17T11:44:16.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-17T11:44:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: pgmap v87: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:44:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/611823776' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi049-76288-6", "pool2": "test-rados-api-smithi049-76288-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi049-75722-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: osdmap e94: 8 total, 8 up, 8 in 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"} : dispatch 2024-09-17T11:44:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:16.612 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:17.061 INFO:tasks.workunit.client.0.smithi049.stdout:nes) 2024-09-17T11:44:17.061 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: data: 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pools: 33 pools, 900 pgs 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: objects: 257 objects, 145 MiB 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: usage: 737 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: pgs: 46.222% pgs unknown 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 7.111% pgs not active 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 420 active+clean 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 416 unknown 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 64 creating+peering 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:44:17.062 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: io: 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: client: 12 MiB/s rd, 39 MiB/s wr, 265 op/s rd, 302 op/s wr 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4268 ms) 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ OK ] LibRadosService.Status (20021 ms) 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [----------] 4 tests from LibRadosService (29486 ms total) 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [----------] Global test environment tear-down 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29486 ms total) 2024-09-17T11:44:17.063 INFO:tasks.workunit.client.0.smithi049.stdout: api_service: [ PASSED ] 4 tests. 2024-09-17T11:44:17.327 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:17.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:16 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:17.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:17.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:17.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:16 smithi129 ceph-mon[33364]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T11:44:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[30017]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T11:44:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-9"}]': finished 2024-09-17T11:44:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3595934388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi049-75736-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:16 smithi049 ceph-mon[40223]: osdmap e95: 8 total, 8 up, 8 in 2024-09-17T11:44:17.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:17.923 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5084 ms) 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (81 ms) 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20064 ms) 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-17T11:44:17.924 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: attempt 0 of 20 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5135 ms) 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30364 ms total) 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30364 ms total) 2024-09-17T11:44:17.925 INFO:tasks.workunit.client.0.smithi049.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: pgmap v90: 516 pgs: 160 unknown, 356 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3461339929' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2506957178' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: pgmap v90: 516 pgs: 160 unknown, 356 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.414 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3461339929' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.415 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.415 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2506957178' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: pgmap v90: 516 pgs: 160 unknown, 356 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi049-76082-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: osdmap e96: 8 total, 8 up, 8 in 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3461339929' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:18.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2506957178' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-17T11:44:19.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2742612551' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:19.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-17T11:44:19.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2742612551' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.405 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi049-75722-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: osdmap e97: 8 total, 8 up, 8 in 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2742612551' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:20.338 INFO:tasks.workunit.client.0.smithi049.stdout: misc: Running main() from gmock_main.cc 2024-09-17T11:44:20.338 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-09-17T11:44:20.338 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [----------] Global test environment set-up. 2024-09-17T11:44:20.338 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-09-17T11:44:20.338 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.Version (1777 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (2004 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.LongName (2623 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3018 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3023 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3014 ms) 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-09-17T11:44:20.339 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.Exec (2980 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3015 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3048 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2946 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2026 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3001 ms) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [----------] 12 tests from NeoRadosMisc (32475 ms total) 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [----------] Global test environment tear-down 2024-09-17T11:44:20.340 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [==========] 12 tests from 1 test suite ran. (32476 ms total) 2024-09-17T11:44:20.341 INFO:tasks.workunit.client.0.smithi049.stdout: misc: [ PASSED ] 12 tests. 2024-09-17T11:44:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: pgmap v93: 556 pgs: 72 creating+peering, 160 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T11:44:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:20.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: pgmap v93: 556 pgs: 72 creating+peering, 160 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:20.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:20.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T11:44:20.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:20.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: pgmap v93: 556 pgs: 72 creating+peering, 160 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi049-75736-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: osdmap e98: 8 total, 8 up, 8 in 2024-09-17T11:44:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""} : dispatch 2024-09-17T11:44:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""}]': finished 2024-09-17T11:44:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T11:44:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4199238996' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:21.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:21.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""}]': finished 2024-09-17T11:44:21.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T11:44:21.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4199238996' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:21.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache", "force_nonempty":""}]': finished 2024-09-17T11:44:21.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: osdmap e99: 8 total, 8 up, 8 in 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4199238996' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: pgmap v96: 492 pgs: 40 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.002 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.003 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/635183575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.003 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.003 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:44:22.003 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:44:22.003 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:44:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: pgmap v96: 492 pgs: 40 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:44:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/635183575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:44:22.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:44:22.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:21 smithi049 ceph-mon[30017]: pgmap v96: 492 pgs: 40 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi049-75722-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:22.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: osdmap e100: 8 total, 8 up, 8 in 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/986216340' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/635183575' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.0"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.1"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.2"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:44:22.407 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:44:22.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:44:22.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:44:22.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:44:22.408 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:44:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.3"} : dispatch 2024-09-17T11:44:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.4"} : dispatch 2024-09-17T11:44:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.5"} : dispatch 2024-09-17T11:44:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.6"} : dispatch 2024-09-17T11:44:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.7"} : dispatch 2024-09-17T11:44:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.8"} : dispatch 2024-09-17T11:44:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "15.9"} : dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:44:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: 15.3 deep-scrub starts 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: 15.3 deep-scrub ok 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"}]': finished 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T11:44:23.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:44:23.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: 15.3 deep-scrub starts 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: 15.3 deep-scrub ok 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"}]': finished 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T11:44:23.406 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:23.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-09-17T11:44:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-09-17T11:44:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-09-17T11:44:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-09-17T11:44:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-09-17T11:44:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-09-17T11:44:23.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-09-17T11:44:23.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-09-17T11:44:23.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-09-17T11:44:23.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-09-17T11:44:23.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: 15.3 deep-scrub starts 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: 15.3 deep-scrub ok 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76009-10", "tierpool":"test-rados-api-smithi049-76009-10-cache"}]': finished 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi049-75736-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: osdmap e101: 8 total, 8 up, 8 in 2024-09-17T11:44:23.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"} : dispatch 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: Running main() from gmock_main.cc 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [----------] Global test environment set-up. 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-17T11:44:24.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2226 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3010 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (2638 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3011 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3058 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4954 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5035 ms) 2024-09-17T11:44:24.315 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4986 ms) 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7992 ms) 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [----------] 9 tests from LibRadosPools (36910 ms total) 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [----------] Global test environment tear-down 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (36911 ms total) 2024-09-17T11:44:24.316 INFO:tasks.workunit.client.0.smithi049.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-17T11:44:24.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.0 deep-scrub starts 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.0 deep-scrub ok 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.4 deep-scrub starts 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.4 deep-scrub ok 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.6 deep-scrub starts 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.6 deep-scrub ok 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: pgmap v99: 556 pgs: 1 active+clean+scrubbing+deep, 555 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 10 op/s 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.1 deep-scrub starts 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: 15.1 deep-scrub ok 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"}]': finished 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/224346515' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.0 deep-scrub starts 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.0 deep-scrub ok 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.4 deep-scrub starts 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.4 deep-scrub ok 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.6 deep-scrub starts 2024-09-17T11:44:24.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.6 deep-scrub ok 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: pgmap v99: 556 pgs: 1 active+clean+scrubbing+deep, 555 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 10 op/s 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.1 deep-scrub starts 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: 15.1 deep-scrub ok 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"}]': finished 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/224346515' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:24.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.0 deep-scrub starts 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.0 deep-scrub ok 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.4 deep-scrub starts 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.4 deep-scrub ok 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.6 deep-scrub starts 2024-09-17T11:44:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.6 deep-scrub ok 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: pgmap v99: 556 pgs: 1 active+clean+scrubbing+deep, 555 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 10 op/s 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.1 deep-scrub starts 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: 15.1 deep-scrub ok 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-11", "mode": "writeback"}]': finished 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: osdmap e102: 8 total, 8 up, 8 in 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/224346515' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.2 deep-scrub starts 2024-09-17T11:44:25.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.2 deep-scrub ok 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.7 deep-scrub starts 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.7 deep-scrub ok 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.5 deep-scrub starts 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: 15.5 deep-scrub ok 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3086390206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.2 deep-scrub starts 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.2 deep-scrub ok 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.7 deep-scrub starts 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.7 deep-scrub ok 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.5 deep-scrub starts 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: 15.5 deep-scrub ok 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3086390206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:25.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:25.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.2 deep-scrub starts 2024-09-17T11:44:25.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.2 deep-scrub ok 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.7 deep-scrub starts 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.7 deep-scrub ok 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.5 deep-scrub starts 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: 15.5 deep-scrub ok 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi049-75722-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: osdmap e103: 8 total, 8 up, 8 in 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3086390206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] Global test environment set-up. 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-17T11:44:26.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: unable to get monitor info from DNS SRV with service name: 2024-09-17T11:43:47.484+0000 7f933d233780 -1 failed for service _ceph-mon._tcp 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: ceph-mon 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: 2024-09-17T11:43:47.484+0000 7f933d233780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (87 ms) 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5016 ms) 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5103 ms total) 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: started 0x7f932406a2b0 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: started 0x55ec51621150 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: started 2 aios 2024-09-17T11:44:26.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: waiting 0x7f932406a2b0 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: waiting 0x55ec51621150 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: done. 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (4773 ms) 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (4773 ms total) 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.Exec (101 ms) 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (32 ms) 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-17T11:44:26.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (5 ms) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4840 ms) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (21480 ms) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] 8 tests from LibRadosMisc (26458 ms total) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [----------] Global test environment tear-down 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (38895 ms total) 2024-09-17T11:44:26.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: 15.8 deep-scrub starts 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: 15.8 deep-scrub ok 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: pgmap v102: 492 pgs: 128 unknown, 1 active+clean+scrubbing+deep, 363 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: 15.9 deep-scrub starts 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: 15.9 deep-scrub ok 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: 15.8 deep-scrub starts 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: 15.8 deep-scrub ok 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: pgmap v102: 492 pgs: 128 unknown, 1 active+clean+scrubbing+deep, 363 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: 15.9 deep-scrub starts 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: 15.9 deep-scrub ok 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:26.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:26.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T11:44:26.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: 15.8 deep-scrub starts 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: 15.8 deep-scrub ok 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: pgmap v102: 492 pgs: 128 unknown, 1 active+clean+scrubbing+deep, 363 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: 15.9 deep-scrub starts 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: 15.9 deep-scrub ok 2024-09-17T11:44:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi049-75736-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:44:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: osdmap e104: 8 total, 8 up, 8 in 2024-09-17T11:44:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"} : dispatch 2024-09-17T11:44:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:27.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:44:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-11"}]': finished 2024-09-17T11:44:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: osdmap e105: 8 total, 8 up, 8 in 2024-09-17T11:44:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:27.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: pgmap v105: 396 pgs: 64 unknown, 1 active+clean+scrubbing+deep, 331 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2162849532' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: pgmap v105: 396 pgs: 64 unknown, 1 active+clean+scrubbing+deep, 331 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2162849532' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: pgmap v105: 396 pgs: 64 unknown, 1 active+clean+scrubbing+deep, 331 active+clean; 217 MiB data, 982 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:28.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1282524275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi049-75722-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: osdmap e106: 8 total, 8 up, 8 in 2024-09-17T11:44:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2162849532' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.304 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: Running main() from gmock_main.cc 2024-09-17T11:44:29.304 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-09-17T11:44:29.304 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [----------] Global test environment set-up. 2024-09-17T11:44:29.304 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-09-17T11:44:29.304 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2729 ms) 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3046 ms) 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (2618 ms) 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3011 ms) 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3010 ms) 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-09-17T11:44:29.305 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (2973 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3031 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3033 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2953 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3006 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3055 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2987 ms) 2024-09-17T11:44:29.306 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2945 ms) 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3015 ms) 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41412 ms total) 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [----------] Global test environment tear-down 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41412 ms total) 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: read_operations: [ PASSED ] 14 tests. 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: io: Running main() from gmock_main.cc 2024-09-17T11:44:29.307 INFO:tasks.workunit.client.0.smithi049.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [----------] Global test environment set-up. 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [----------] 14 tests from NeoRadosIo 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.Limits (2799 ms) 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3041 ms) 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.ReadOp (2626 ms) 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.SparseRead (3001 ms) 2024-09-17T11:44:29.308 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3023 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (2988 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3010 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3014 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2971 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.Trunc (3016 ms) 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-09-17T11:44:29.309 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.Remove (3017 ms) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3014 ms) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.RmXattr (2958 ms) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3007 ms) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [----------] 14 tests from NeoRadosIo (41485 ms total) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [----------] Global test environment tear-down 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [==========] 14 tests from 1 test suite ran. (41486 ms total) 2024-09-17T11:44:29.310 INFO:tasks.workunit.client.0.smithi049.stdout: io: [ PASSED ] 14 tests. 2024-09-17T11:44:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: osdmap e107: 8 total, 8 up, 8 in 2024-09-17T11:44:29.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3770511399' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"} : dispatch 2024-09-17T11:44:29.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:29.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: pgmap v108: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T11:44:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/155838636' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: pgmap v108: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/155838636' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:30.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: pgmap v108: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi049-76082-10"}]': finished 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: osdmap e108: 8 total, 8 up, 8 in 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/155838636' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:30.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:30.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi049-75722-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: osdmap e109: 8 total, 8 up, 8 in 2024-09-17T11:44:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[30017]: pgmap v111: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-09-17T11:44:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[30017]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T11:44:32.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[40223]: pgmap v111: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-09-17T11:44:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:32 smithi049 ceph-mon[40223]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T11:44:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:32 smithi129 ceph-mon[33364]: pgmap v111: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-09-17T11:44:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/842722436' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi049-75736-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:32 smithi129 ceph-mon[33364]: osdmap e110: 8 total, 8 up, 8 in 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[40223]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:33.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:33.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[30017]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T11:44:33.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi049-76082-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:33 smithi129 ceph-mon[33364]: osdmap e111: 8 total, 8 up, 8 in 2024-09-17T11:44:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: pgmap v114: 364 pgs: 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/964365816' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: pgmap v114: 364 pgs: 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/964365816' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: pgmap v114: 364 pgs: 40 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1605131481' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi049-75722-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: osdmap e112: 8 total, 8 up, 8 in 2024-09-17T11:44:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/964365816' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi049-75736-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:44:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: osdmap e113: 8 total, 8 up, 8 in 2024-09-17T11:44:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:44:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: pgmap v117: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:44:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:44:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T11:44:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: pgmap v117: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: pgmap v117: 396 pgs: 64 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 97 KiB/s wr, 97 op/s 2024-09-17T11:44:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: osdmap e114: 8 total, 8 up, 8 in 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"} : dispatch 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:36.894 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:37.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:37.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"}]': finished 2024-09-17T11:44:37.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:37.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2614945267' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-17T11:44:37.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"}]': finished 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2614945267' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:37.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-17T11:44:37.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-17T11:44:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-13", "mode": "writeback"}]': finished 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/689956641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi049-75722-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: osdmap e115: 8 total, 8 up, 8 in 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2614945267' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.0"} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.1"} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.2"} : dispatch 2024-09-17T11:44:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.3"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.4"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.5"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.6"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.7"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.8"} : dispatch 2024-09-17T11:44:37.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "171.9"} : dispatch 2024-09-17T11:44:37.824 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: pgmap v120: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-17T11:44:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-17T11:44:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-17T11:44:38.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.4 scrub starts 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.4 scrub ok 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.2 scrub starts 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.2 scrub ok 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.8 scrub starts 2024-09-17T11:44:38.179 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:38 smithi129 ceph-mon[33364]: 171.8 scrub ok 2024-09-17T11:44:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: pgmap v120: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.4 scrub starts 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.4 scrub ok 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.2 scrub starts 2024-09-17T11:44:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.2 scrub ok 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.8 scrub starts 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[40223]: 171.8 scrub ok 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: pgmap v120: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-09-17T11:44:38.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.4 scrub starts 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.4 scrub ok 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.2 scrub starts 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.2 scrub ok 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi049-75736-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: osdmap e116: 8 total, 8 up, 8 in 2024-09-17T11:44:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:38.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.8 scrub starts 2024-09-17T11:44:38.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:38 smithi049 ceph-mon[30017]: 171.8 scrub ok 2024-09-17T11:44:39.332 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:39.332 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.6 scrub starts 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.6 scrub ok 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.0 scrub starts 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.0 scrub ok 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.5 scrub starts 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.5 scrub ok 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.3 scrub starts 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.3 scrub ok 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:39.333 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.9 scrub starts 2024-09-17T11:44:39.334 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:39 smithi129 ceph-mon[33364]: 171.9 scrub ok 2024-09-17T11:44:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.6 scrub starts 2024-09-17T11:44:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.6 scrub ok 2024-09-17T11:44:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.0 scrub starts 2024-09-17T11:44:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.0 scrub ok 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.5 scrub starts 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.5 scrub ok 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.3 scrub starts 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.3 scrub ok 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.9 scrub starts 2024-09-17T11:44:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[30017]: 171.9 scrub ok 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.6 scrub starts 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.6 scrub ok 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.0 scrub starts 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.0 scrub ok 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.5 scrub starts 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.5 scrub ok 2024-09-17T11:44:39.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.3 scrub starts 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.3 scrub ok 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: osdmap e117: 8 total, 8 up, 8 in 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.9 scrub starts 2024-09-17T11:44:39.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:39 smithi049 ceph-mon[40223]: 171.9 scrub ok 2024-09-17T11:44:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: pgmap v123: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: 171.1 scrub starts 2024-09-17T11:44:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: 171.1 scrub ok 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1643295524' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: pgmap v123: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: 171.1 scrub starts 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: 171.1 scrub ok 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1643295524' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:44:40.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:40 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: pgmap v123: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: 171.1 scrub starts 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: 171.1 scrub ok 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4012686847' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi049-75722-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: osdmap e118: 8 total, 8 up, 8 in 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1643295524' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:44:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:40 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: 171.7 scrub starts 2024-09-17T11:44:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: 171.7 scrub ok 2024-09-17T11:44:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[40223]: pgmap v126: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: 171.7 scrub starts 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: 171.7 scrub ok 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:41 smithi049 ceph-mon[30017]: pgmap v126: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: 171.7 scrub starts 2024-09-17T11:44:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: 171.7 scrub ok 2024-09-17T11:44:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi049-75736-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: osdmap e119: 8 total, 8 up, 8 in 2024-09-17T11:44:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:41 smithi129 ceph-mon[33364]: pgmap v126: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 31 KiB/s wr, 103 op/s 2024-09-17T11:44:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T11:44:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1288488733' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1288488733' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:42.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: osdmap e120: 8 total, 8 up, 8 in 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1288488733' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:44:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:43.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-17T11:44:43.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2003 ms) 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2005 ms) 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (1628 ms) 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1980 ms) 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (7617 ms total) 2024-09-17T11:44:43.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4046 ms) 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4047 ms) 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4999 ms) 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13092 ms total) 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3018 ms) 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-17T11:44:43.306 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2020 ms) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1935 ms) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2007 ms) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8980 ms total) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3987 ms) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4027 ms) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8014 ms total) 2024-09-17T11:44:43.307 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: 2024-09-17T11:44:43.308 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-17T11:44:43.308 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55869 ms total) 2024-09-17T11:44:43.308 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-09-17T11:44:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: pgmap v129: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:43.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: pgmap v129: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:43.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi049-75722-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: osdmap e121: 8 total, 8 up, 8 in 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4081080083' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"} : dispatch 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: pgmap v129: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1175199844' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi049-75736-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T11:44:44.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3174017058' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1288402967' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3174017058' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1288402967' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi049-76082-15"}]': finished 2024-09-17T11:44:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: osdmap e122: 8 total, 8 up, 8 in 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3174017058' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76109-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: osdmap e123: 8 total, 8 up, 8 in 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1288402967' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: pgmap v132: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T11:44:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3917234333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: pgmap v132: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T11:44:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3917234333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: pgmap v132: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75722-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: osdmap e124: 8 total, 8 up, 8 in 2024-09-17T11:44:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3917234333' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:47.582 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:47.583 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:47 smithi129 ceph-mon[33364]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T11:44:47.583 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:47.583 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:47 smithi129 ceph-mon[33364]: pgmap v135: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[30017]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[30017]: pgmap v135: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi049-75736-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[40223]: osdmap e125: 8 total, 8 up, 8 in 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:47 smithi049 ceph-mon[40223]: pgmap v135: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:47.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T11:44:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1010494176' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1010494176' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: osdmap e126: 8 total, 8 up, 8 in 2024-09-17T11:44:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1010494176' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T11:44:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[30017]: pgmap v138: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:49 smithi049 ceph-mon[40223]: pgmap v138: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi049-75722-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: osdmap e127: 8 total, 8 up, 8 in 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:49 smithi129 ceph-mon[33364]: pgmap v138: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[40223]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T11:44:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:50 smithi049 ceph-mon[30017]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T11:44:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:50 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1988037119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi049-75736-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:50 smithi129 ceph-mon[33364]: osdmap e128: 8 total, 8 up, 8 in 2024-09-17T11:44:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:44:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T11:44:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/707801650' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[30017]: pgmap v141: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/707801650' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:51 smithi049 ceph-mon[40223]: pgmap v141: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi049-76109-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:44:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: osdmap e129: 8 total, 8 up, 8 in 2024-09-17T11:44:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/707801650' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:51 smithi129 ceph-mon[33364]: pgmap v141: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:44:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T11:44:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3873762131' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3873762131' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi049-75722-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: osdmap e130: 8 total, 8 up, 8 in 2024-09-17T11:44:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3873762131' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-17T11:44:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2"} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: pgmap v144: 364 pgs: 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2729202310' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2"} : dispatch 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: pgmap v144: 364 pgs: 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2729202310' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: osdmap e131: 8 total, 8 up, 8 in 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2"} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: pgmap v144: 364 pgs: 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi049-75964-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: osdmap e132: 8 total, 8 up, 8 in 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2729202310' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:53.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-17T11:44:55.283 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: seed 76109 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2013 ms) 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2018 ms) 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (1627 ms) 2024-09-17T11:44:55.284 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2008 ms) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4041 ms) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11707 ms total) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4022 ms) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4002 ms) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5963 ms) 2024-09-17T11:44:55.285 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4031 ms) 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2110 ms) 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4937-g74984932/rpm/el9/BUILD/ceph-19.3.0-4937-g74984932/src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi049-76109-7 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17862 ms) 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37990 ms total) 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:44:55.286 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5044 ms) 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5053 ms total) 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3011 ms) 2024-09-17T11:44:55.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-17T11:44:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1502411844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[40223]: pgmap v147: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1502411844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:55 smithi049 ceph-mon[30017]: pgmap v147: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:44:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi049-75722-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: osdmap e133: 8 total, 8 up, 8 in 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1502411844' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:44:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:55 smithi129 ceph-mon[33364]: pgmap v147: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T11:44:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:44:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi049-75736-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: osdmap e134: 8 total, 8 up, 8 in 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3851778625' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"} : dispatch 2024-09-17T11:44:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:44:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:44:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:44:57.287 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnaps api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: seed 75964 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-17T11:44:57.288 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (3 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (210 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (11 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (6 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-09-17T11:44:57.289 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (4 ms) 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-17T11:44:57.290 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (6 ms) 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1635 ms) 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (509 ms) 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: done waiting, doing copies 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-17T11:44:57.291 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: done waiting 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60707 ms) 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4555 ms) 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-17T11:44:57.292 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-17T11:44:57.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-17T11:44:57.293 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (1 ms) 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: pgmap v150: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T11:44:57.594 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: pgmap v150: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi049-75964-1","app":"app1","key":"key1"}]': finished 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: osdmap e135: 8 total, 8 up, 8 in 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: pgmap v150: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:44:57.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3413705064' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi049-75722-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:57.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: osdmap e136: 8 total, 8 up, 8 in 2024-09-17T11:44:57.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:57.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:44:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:44:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2808955453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2808955453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:58.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:58.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T11:44:58.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:58.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2808955453' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi049-75736-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-75964-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: osdmap e137: 8 total, 8 up, 8 in 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:58.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:44:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[40223]: pgmap v153: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[40223]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T11:44:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[30017]: pgmap v153: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:44:59 smithi049 ceph-mon[30017]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T11:44:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:59 smithi129 ceph-mon[33364]: pgmap v153: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:44:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:44:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:44:59 smithi129 ceph-mon[33364]: osdmap e138: 8 total, 8 up, 8 in 2024-09-17T11:45:00.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2110577129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T11:45:00.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2110577129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-75964-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4002408023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi049-75722-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: osdmap e139: 8 total, 8 up, 8 in 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2110577129' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:00.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: pgmap v156: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:45:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: pgmap v156: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: pgmap v156: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-17T11:45:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi049-75736-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: osdmap e140: 8 total, 8 up, 8 in 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2361728393' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"} : dispatch 2024-09-17T11:45:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 snapshots: Running main() from gmock_main.cc 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] Global test environment set-up. 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-09-17T11:45:02.305 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4741 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (4649 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4018 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (4983 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7045 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25436 ms total) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5008 ms) 2024-09-17T11:45:02.306 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5939 ms) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8031 ms) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5991 ms) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4007 ms) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi049-77130-11. 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: Waiting for snaps to purge. 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19998 ms) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48974 ms total) 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: 2024-09-17T11:45:02.307 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [----------] Global test environment tear-down 2024-09-17T11:45:02.308 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (74410 ms total) 2024-09-17T11:45:02.308 INFO:tasks.workunit.client.0.smithi049.stdout: snapshots: [ PASSED ] 11 tests. 2024-09-17T11:45:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T11:45:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1151881064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1151881064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi049-76109-16"}]': finished 2024-09-17T11:45:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: osdmap e141: 8 total, 8 up, 8 in 2024-09-17T11:45:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1151881064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:02.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: pgmap v159: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T11:45:03.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: pgmap v159: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:03.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: pgmap v159: 332 pgs: 32 creating+peering, 300 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75964-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi049-75722-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:03.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: osdmap e142: 8 total, 8 up, 8 in 2024-09-17T11:45:03.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:03.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:03.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T11:45:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[40223]: pgmap v162: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T11:45:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:05 smithi049 ceph-mon[30017]: pgmap v162: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:05.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/684375801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi049-75736-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: osdmap e143: 8 total, 8 up, 8 in 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:05 smithi129 ceph-mon[33364]: pgmap v162: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:06.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi049-75722-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: osdmap e144: 8 total, 8 up, 8 in 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:06.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:07.604 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: pgmap v165: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:07.605 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T11:45:07.606 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: pgmap v165: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2923891004' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: osdmap e145: 8 total, 8 up, 8 in 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"} : dispatch 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: pgmap v165: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi049-75722-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-75964-24"}]': finished 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2111127642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi049-75736-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:07.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: osdmap e146: 8 total, 8 up, 8 in 2024-09-17T11:45:07.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:07.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[40223]: osdmap e147: 8 total, 8 up, 8 in 2024-09-17T11:45:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:08 smithi049 ceph-mon[30017]: osdmap e147: 8 total, 8 up, 8 in 2024-09-17T11:45:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:08 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:08 smithi129 ceph-mon[33364]: osdmap e147: 8 total, 8 up, 8 in 2024-09-17T11:45:09.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1321022114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: pgmap v168: 308 pgs: 8 creating+peering, 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: osdmap e148: 8 total, 8 up, 8 in 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1420842642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1321022114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: pgmap v168: 308 pgs: 8 creating+peering, 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: osdmap e148: 8 total, 8 up, 8 in 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1420842642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1321022114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: pgmap v168: 308 pgs: 8 creating+peering, 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi049-75964-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: osdmap e148: 8 total, 8 up, 8 in 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1420842642' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: Running main() from gmock_main.cc 2024-09-17T11:45:11.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-09-17T11:45:11.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] Global test environment set-up. 2024-09-17T11:45:11.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-09-17T11:45:11.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-17T11:45:11.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2352 ms) 2024-09-17T11:45:11.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3021 ms) 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2629 ms) 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3006 ms) 2024-09-17T11:45:11.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3043 ms) 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2954 ms) 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3020 ms) 2024-09-17T11:45:11.335 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-17T11:45:11.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3015 ms) 2024-09-17T11:45:11.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-17T11:45:11.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3038 ms) 2024-09-17T11:45:11.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-17T11:45:11.336 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2957 ms) 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3045 ms) 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3015 ms) 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-17T11:45:11.337 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2933 ms) 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2994 ms) 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.Flush (3025 ms) 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-17T11:45:11.338 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2957 ms) 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3017 ms) 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3001 ms) 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-17T11:45:11.339 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3024 ms) 2024-09-17T11:45:11.340 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-17T11:45:11.340 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (2995 ms) 2024-09-17T11:45:11.340 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-09-17T11:45:11.340 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3029 ms) 2024-09-17T11:45:11.340 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3008 ms) 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2940 ms) 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3005 ms) 2024-09-17T11:45:11.341 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3010 ms) 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3016 ms) 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] 26 tests from LibRadosAio (77050 ms total) 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: 2024-09-17T11:45:11.342 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-17T11:45:11.343 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-17T11:45:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: osdmap e149: 8 total, 8 up, 8 in 2024-09-17T11:45:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: pgmap v171: 300 pgs: 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: osdmap e149: 8 total, 8 up, 8 in 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: pgmap v171: 300 pgs: 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:11.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi049-75736-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: osdmap e149: 8 total, 8 up, 8 in 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/655303957' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"} : dispatch 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: pgmap v171: 300 pgs: 32 unknown, 268 active+clean; 607 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:12.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:12.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:12.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: osdmap e150: 8 total, 8 up, 8 in 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4260381273' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: osdmap e150: 8 total, 8 up, 8 in 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4260381273' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi049-75722-27"}]': finished 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: osdmap e150: 8 total, 8 up, 8 in 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4260381273' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:12.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:12.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3238805579' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: pgmap v174: 332 pgs: 32 creating+peering, 300 active+clean; 618 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3238805579' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: pgmap v174: 332 pgs: 32 creating+peering, 300 active+clean; 618 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-09-17T11:45:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-13"}]': finished 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi049-75964-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi049-75722-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: osdmap e151: 8 total, 8 up, 8 in 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3238805579' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: pgmap v174: 332 pgs: 32 creating+peering, 300 active+clean; 618 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi049-75736-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T11:45:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T11:45:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: osdmap e152: 8 total, 8 up, 8 in 2024-09-17T11:45:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi049-75722-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: osdmap e153: 8 total, 8 up, 8 in 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: pgmap v177: 308 pgs: 72 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-17T11:45:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/736098728' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: pgmap v177: 308 pgs: 72 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T11:45:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/736098728' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: pgmap v177: 308 pgs: 72 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-17T11:45:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2911995388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi049-75964-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: osdmap e154: 8 total, 8 up, 8 in 2024-09-17T11:45:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/736098728' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:17.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:17.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T11:45:17.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.617 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.617 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.617 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.617 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:17.617 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:17.618 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-mon[33364]: pgmap v180: 300 pgs: 64 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T11:45:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[30017]: pgmap v180: 300 pgs: 64 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi049-75736-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: osdmap e155: 8 total, 8 up, 8 in 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:17.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:17 smithi049 ceph-mon[40223]: pgmap v180: 300 pgs: 64 unknown, 236 active+clean; 610 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:17.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T11:45:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: osdmap e156: 8 total, 8 up, 8 in 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2687930625' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:18.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"} : dispatch 2024-09-17T11:45:18.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:18.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/752543626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: pgmap v183: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/752543626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: pgmap v183: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"}]': finished 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:19.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"}]': finished 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:19.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:19.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T11:45:19.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi049-75722-28"}]': finished 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: osdmap e157: 8 total, 8 up, 8 in 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/752543626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: pgmap v183: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi049-75964-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-15", "mode": "writeback"}]': finished 2024-09-17T11:45:19.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi049-75736-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi049-75722-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3911233488' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: osdmap e158: 8 total, 8 up, 8 in 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"} : dispatch 2024-09-17T11:45:19.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:20.303 INFO:tasks.workunit.client.0.smithi049.stdout: hotsECPP.SnapRemovePP (1940 ms) 2024-09-17T11:45:20.303 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-17T11:45:20.303 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2003 ms) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2012 ms) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8967 ms total) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4043 ms) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3988 ms) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3951 ms) 2024-09-17T11:45:20.304 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11982 ms total) 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (92871 ms total) 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-09-17T11:45:20.305 INFO:tasks.workunit.client.0.smithi049.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-17T11:45:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[40223]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T11:45:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:20 smithi049 ceph-mon[30017]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T11:45:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi049-76109-21"}]': finished 2024-09-17T11:45:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:20 smithi129 ceph-mon[33364]: osdmap e159: 8 total, 8 up, 8 in 2024-09-17T11:45:21.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:21.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: pgmap v186: 268 pgs: 8 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1261501408' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: pgmap v186: 268 pgs: 8 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1261501408' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: pgmap v186: 268 pgs: 8 unknown, 260 active+clean; 586 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi049-75722-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: osdmap e160: 8 total, 8 up, 8 in 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1261501408' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"} : dispatch 2024-09-17T11:45:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T11:45:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T11:45:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi049-75736-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-15"}]': finished 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: osdmap e161: 8 total, 8 up, 8 in 2024-09-17T11:45:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"} : dispatch 2024-09-17T11:45:23.311 INFO:tasks.workunit.client.0.smithi049.stdout:7683 ms total) 2024-09-17T11:45:23.311 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (107 ms) 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (107 ms total) 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (41 ms) 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (8 ms) 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (49 ms total) 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.312 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (16 ms) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (2 ms) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (18 ms total) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (36 ms) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (5 ms) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (41 ms total) 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-17T11:45:23.313 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1035 ms) 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1035 ms total) 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (95919 ms total) 2024-09-17T11:45:23.314 INFO:tasks.workunit.client.0.smithi049.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-17T11:45:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: pgmap v189: 300 pgs: 300 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: pgmap v189: 300 pgs: 300 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T11:45:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:23.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: pgmap v189: 300 pgs: 300 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3960856575' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi049-75964-36"}]': finished 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: osdmap e162: 8 total, 8 up, 8 in 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/742308635' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/742308635' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: osdmap e163: 8 total, 8 up, 8 in 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/584045880' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/742308635' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:24.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:24.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: pgmap v192: 292 pgs: 64 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: pgmap v192: 292 pgs: 64 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:25.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: pgmap v192: 292 pgs: 64 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi049-75722-29"}]': finished 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi049-75736-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pg_num","val":"11"}]': finished 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: osdmap e164: 8 total, 8 up, 8 in 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:25.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:26.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:26.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:26.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:26.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:26.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi049-75804-1","var":"pgp_num","val":"11"}]': finished 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi049-75722-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: osdmap e165: 8 total, 8 up, 8 in 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:27.626 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: pgmap v195: 260 pgs: 32 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.627 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: pgmap v195: 260 pgs: 32 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T11:45:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: pgmap v195: 260 pgs: 32 unknown, 228 active+clean; 586 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: osdmap e166: 8 total, 8 up, 8 in 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"} : dispatch 2024-09-17T11:45:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:27.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"}]': finished 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T11:45:28.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"}]': finished 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T11:45:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi049-75722-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-17", "mode": "writeback"}]': finished 2024-09-17T11:45:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3917176136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi049-75736-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: osdmap e167: 8 total, 8 up, 8 in 2024-09-17T11:45:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-17T11:45:29.297 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2356 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (21704 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6026 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2982 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3001 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2945 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3018 ms) 2024-09-17T11:45:29.298 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3011 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2966 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3008 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3011 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3008 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3019 ms) 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-17T11:45:29.299 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3010 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3012 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2938 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3016 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3003 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3009 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3020 ms) 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-17T11:45:29.300 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2999 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (2949 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3012 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3016 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3005 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2977 ms) 2024-09-17T11:45:29.301 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-17T11:45:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: pgmap v198: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: pgmap v198: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:29.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:29.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: pgmap v198: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: osdmap e168: 8 total, 8 up, 8 in 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"} : dispatch 2024-09-17T11:45:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T11:45:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1483265859' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1483265859' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:30.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-17"}]': finished 2024-09-17T11:45:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: osdmap e169: 8 total, 8 up, 8 in 2024-09-17T11:45:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1483265859' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: pgmap v201: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: pgmap v201: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:31.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: pgmap v201: 292 pgs: 32 unknown, 260 active+clean; 586 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-17T11:45:31.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi049-75736-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: osdmap e170: 8 total, 8 up, 8 in 2024-09-17T11:45:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1753643902' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"} : dispatch 2024-09-17T11:45:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T11:45:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[30017]: pgmap v204: 259 pgs: 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:33.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:33 smithi049 ceph-mon[40223]: pgmap v204: 259 pgs: 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:33.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi049-75722-30"}]': finished 2024-09-17T11:45:33.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: osdmap e171: 8 total, 8 up, 8 in 2024-09-17T11:45:33.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:33 smithi129 ceph-mon[33364]: pgmap v204: 259 pgs: 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:34.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:34.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:34.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2484547570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2484547570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:34 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:34.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi049-75722-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: osdmap e172: 8 total, 8 up, 8 in 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2484547570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:34 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T11:45:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: pgmap v207: 291 pgs: 32 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: pgmap v207: 291 pgs: 32 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:35.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi049-75736-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: osdmap e173: 8 total, 8 up, 8 in 2024-09-17T11:45:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: pgmap v207: 291 pgs: 32 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi049-75722-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: osdmap e174: 8 total, 8 up, 8 in 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:35.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"} : dispatch 2024-09-17T11:45:36.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"}]': finished 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/321722746' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"}]': finished 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T11:45:36.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:36.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/321722746' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-19", "mode": "writeback"}]': finished 2024-09-17T11:45:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: osdmap e175: 8 total, 8 up, 8 in 2024-09-17T11:45:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/321722746' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:37.630 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.630 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.630 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: pgmap v210: 299 pgs: 40 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:37.630 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:37.630 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:37.631 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: pgmap v210: 299 pgs: 40 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T11:45:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: pgmap v210: 299 pgs: 40 unknown, 32 creating+peering, 227 active+clean; 586 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi049-75736-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: osdmap e176: 8 total, 8 up, 8 in 2024-09-17T11:45:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"} : dispatch 2024-09-17T11:45:37.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:37.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:37.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:38.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-19"}]': finished 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: osdmap e177: 8 total, 8 up, 8 in 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/573444814' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"} : dispatch 2024-09-17T11:45:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: pgmap v213: 259 pgs: 259 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: pgmap v213: 259 pgs: 259 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-17T11:45:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T11:45:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:39.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: pgmap v213: 259 pgs: 259 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:45:39.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi049-75722-31"}]': finished 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi049-75804-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: osdmap e178: 8 total, 8 up, 8 in 2024-09-17T11:45:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T11:45:40.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi049-75736-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi049-75722-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2510334236' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: osdmap e179: 8 total, 8 up, 8 in 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:40.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:41.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:45:41.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: pgmap v216: 291 pgs: 64 unknown, 227 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={ 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "prefix": "osd pool set", 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "var": "eio", 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "val": "true" 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: } : dispatch 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{ 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "prefix": "osd pool set", 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "var": "eio", 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: "val": "true" 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: }]': finished 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: pgmap v216: 291 pgs: 64 unknown, 227 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:45:41.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={ 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "prefix": "osd pool set", 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "var": "eio", 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "val": "true" 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: } : dispatch 2024-09-17T11:45:41.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{ 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "prefix": "osd pool set", 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "var": "eio", 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: "val": "true" 2024-09-17T11:45:41.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: }]': finished 2024-09-17T11:45:41.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T11:45:41.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:41.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:45:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: pgmap v216: 291 pgs: 64 unknown, 227 active+clean; 586 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd={ 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "prefix": "osd pool set", 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "var": "eio", 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "val": "true" 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: } : dispatch 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3729189663' entity='client.admin' cmd='[{ 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "prefix": "osd pool set", 2024-09-17T11:45:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "pool": "PoolEIOFlag_smithi049-75736-33", 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "var": "eio", 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: "val": "true" 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: }]': finished 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: osdmap e180: 8 total, 8 up, 8 in 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:41.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi049-75722-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi049-75804-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: osdmap e181: 8 total, 8 up, 8 in 2024-09-17T11:45:42.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:42.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:42.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: pgmap v219: 236 pgs: 8 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2124328760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: pgmap v219: 236 pgs: 8 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2124328760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: pgmap v219: 236 pgs: 8 creating+peering, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:45:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: osdmap e182: 8 total, 8 up, 8 in 2024-09-17T11:45:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2124328760' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"} : dispatch 2024-09-17T11:45:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:44.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:44.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"}]': finished 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"}]': finished 2024-09-17T11:45:44.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:44.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T11:45:44.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:44.654 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi049-75804-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-21", "mode": "writeback"}]': finished 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi049-75736-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: osdmap e183: 8 total, 8 up, 8 in 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.549 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAiod=5 2024-09-17T11:45:45.549 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:89d3ae78:::11:head expected=12:89d3ae78:::11:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=11 expected=11 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:863748b0:::15:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:863748b0:::15:head expected=12:863748b0:::15:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=15 expected=15 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:6cac518f:::0:head expected=12:6cac518f:::0:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=0 expected=0 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:62a1935d:::14:head expected=12:62a1935d:::14:head 2024-09-17T11:45:45.550 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=14 expected=14 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:5c6b0b28:::7:head expected=12:5c6b0b28:::7:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=7 expected=7 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:566253c9:::13:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:566253c9:::13:head expected=12:566253c9:::13:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=13 expected=13 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:52ea6a34:::10:head expected=12:52ea6a34:::10:head 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-09-17T11:45:45.551 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=10 expected=10 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : cursor()=12:02547ec2:::1:head expected=12:02547ec2:::1:head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: : entry=1 expected=1 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (191 ms) 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (95144 ms) 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: split 0/5 -> MIN 12:33333333::::head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: split 1/5 -> 12:33333333::::head 12:66666666::::head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: split 2/5 -> 12:66666666::::head 12:99999999::::head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: split 3/5 -> 12:99999999::::head 12:cccccccc::::head 2024-09-17T11:45:45.552 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: split 4/5 -> 12:cccccccc::::head MAX 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (16050 ms) 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 7 tests from LibRadosList (111787 ms total) 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1185 ms) 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo1,foo2,foo3 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo1 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo2 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo3 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo1,foo4,foo5 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo4 2024-09-17T11:45:45.553 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo5 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo1 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset foo6,foo7 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo7 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: foo6 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo4 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo5 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns2:foo7 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns2:foo6 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: ns1:foo1 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo1 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo2 2024-09-17T11:45:45.554 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: :foo3 2024-09-17T11:45:45.555 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (33 ms) 2024-09-17T11:45:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: pgmap v222: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: pgmap v222: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:45.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: pgmap v222: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: osdmap e184: 8 total, 8 up, 8 in 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2688116690' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"} : dispatch 2024-09-17T11:45:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"} : dispatch 2024-09-17T11:45:46.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:46.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:46.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:46.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:46.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:46.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi049-75722-32"}]': finished 2024-09-17T11:45:46.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-21"}]': finished 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: osdmap e185: 8 total, 8 up, 8 in 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:46.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:47.640 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:47.640 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.641 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T11:45:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: pgmap v225: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1540889351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi049-75736-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi049-75722-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: osdmap e186: 8 total, 8 up, 8 in 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2756069233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"} : dispatch 2024-09-17T11:45:47.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:47.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[30017]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T11:45:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[40223]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T11:45:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi049-75804-2"}]': finished 2024-09-17T11:45:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:48 smithi129 ceph-mon[33364]: osdmap e187: 8 total, 8 up, 8 in 2024-09-17T11:45:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: pgmap v228: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: pgmap v228: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: pgmap v228: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi049-75722-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: osdmap e188: 8 total, 8 up, 8 in 2024-09-17T11:45:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 1 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 10 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 13 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 7 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 14 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 0 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 15 0 2024-09-17T11:45:51.361 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 11 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 5 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 8 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 6 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 3 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 4 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 12 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 9 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 2 0 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (44 ms) 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 3 tests from LibRadosListEC (1262 ms total) 2024-09-17T11:45:51.362 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3015 ms) 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] 1 test from LibRadosListNP (3015 ms total) 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [----------] Global test environment tear-down 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [==========] 11 tests from 3 test suites ran. (124026 ms total) 2024-09-17T11:45:51.363 INFO:tasks.workunit.client.0.smithi049.stdout: api_list: [ PASSED ] 11 tests. 2024-09-17T11:45:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T11:45:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[40223]: pgmap v231: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:51 smithi049 ceph-mon[30017]: pgmap v231: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-75804-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2706691218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: osdmap e189: 8 total, 8 up, 8 in 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/938828875' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:45:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:51 smithi129 ceph-mon[33364]: pgmap v231: 300 pgs: 104 unknown, 196 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:45:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-17T11:45:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T11:45:52.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi049-75804-3","pool2":"test-rados-api-smithi049-75804-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: osdmap e190: 8 total, 8 up, 8 in 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: pgmap v234: 292 pgs: 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"}]': finished 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: pgmap v234: 292 pgs: 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:53.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"}]': finished 2024-09-17T11:45:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2314536955' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: osdmap e191: 8 total, 8 up, 8 in 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1427314650' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"} : dispatch 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"} : dispatch 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: pgmap v234: 292 pgs: 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:45:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-23", "mode": "writeback"}]': finished 2024-09-17T11:45:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T11:45:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1898560386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1898560386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:54.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:54.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T11:45:54.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi049-75722-33"}]': finished 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: osdmap e192: 8 total, 8 up, 8 in 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1898560386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:45:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi049-75736-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:54.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi049-75722-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:45:54.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: osdmap e193: 8 total, 8 up, 8 in 2024-09-17T11:45:54.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:54.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T11:45:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:45:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:45:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:45:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: osdmap e194: 8 total, 8 up, 8 in 2024-09-17T11:45:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"} : dispatch 2024-09-17T11:45:56.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:45:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:45:56.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:56.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[40223]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:56.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:56 smithi049 ceph-mon[30017]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T11:45:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:56 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:45:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi049-75722-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-23"}]': finished 2024-09-17T11:45:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:56 smithi129 ceph-mon[33364]: osdmap e195: 8 total, 8 up, 8 in 2024-09-17T11:45:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[30017]: pgmap v240: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:57.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[30017]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[40223]: pgmap v240: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:57 smithi049 ceph-mon[40223]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:57 smithi129 ceph-mon[33364]: pgmap v240: 268 pgs: 8 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:45:57.651 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:57 smithi129 ceph-mon[33364]: osdmap e196: 8 total, 8 up, 8 in 2024-09-17T11:45:57.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:45:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:45:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T11:45:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:58.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:45:58.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: osdmap e197: 8 total, 8 up, 8 in 2024-09-17T11:45:58.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:45:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: pgmap v243: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:45:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: pgmap v243: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:59.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: pgmap v243: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/532310791' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:45:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: osdmap e198: 8 total, 8 up, 8 in 2024-09-17T11:45:59.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/384837395' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:45:59.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:45:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"} : dispatch 2024-09-17T11:46:01.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3384920050' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[30017]: pgmap v246: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3384920050' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:01 smithi049 ceph-mon[40223]: pgmap v246: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi049-75722-34"}]': finished 2024-09-17T11:46:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: osdmap e199: 8 total, 8 up, 8 in 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3384920050' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:01 smithi129 ceph-mon[33364]: pgmap v246: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:02.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:02.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi049-75722-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: osdmap e200: 8 total, 8 up, 8 in 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:03.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: pgmap v249: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"}]': finished 2024-09-17T11:46:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:03.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:03 smithi129 ceph-mon[33364]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T11:46:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: pgmap v249: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"}]': finished 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[40223]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T11:46:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: osdmap e201: 8 total, 8 up, 8 in 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"} : dispatch 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: pgmap v249: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi049-75722-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-25", "mode": "writeback"}]': finished 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/705559788' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:03 smithi049 ceph-mon[30017]: osdmap e202: 8 total, 8 up, 8 in 2024-09-17T11:46:04.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2129238626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2129238626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: osdmap e203: 8 total, 8 up, 8 in 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2129238626' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:05.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: pgmap v252: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: pgmap v252: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: pgmap v252: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-25"}]': finished 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: osdmap e204: 8 total, 8 up, 8 in 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: osdmap e205: 8 total, 8 up, 8 in 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3785827462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: osdmap e205: 8 total, 8 up, 8 in 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3785827462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: osdmap e205: 8 total, 8 up, 8 in 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3275792381' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3785827462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"} : dispatch 2024-09-17T11:46:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: pgmap v255: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: osdmap e206: 8 total, 8 up, 8 in 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: pgmap v255: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: osdmap e206: 8 total, 8 up, 8 in 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: pgmap v255: 356 pgs: 64 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi049-75722-35"}]': finished 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi049-75736-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: osdmap e206: 8 total, 8 up, 8 in 2024-09-17T11:46:07.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:07.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:07.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:08 smithi129 ceph-mon[33364]: osdmap e207: 8 total, 8 up, 8 in 2024-09-17T11:46:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[40223]: osdmap e207: 8 total, 8 up, 8 in 2024-09-17T11:46:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi049-75722-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[30017]: osdmap e207: 8 total, 8 up, 8 in 2024-09-17T11:46:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[40223]: pgmap v258: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[40223]: osdmap e208: 8 total, 8 up, 8 in 2024-09-17T11:46:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[30017]: pgmap v258: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[30017]: osdmap e208: 8 total, 8 up, 8 in 2024-09-17T11:46:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:09 smithi129 ceph-mon[33364]: pgmap v258: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:09 smithi129 ceph-mon[33364]: osdmap e208: 8 total, 8 up, 8 in 2024-09-17T11:46:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:11.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:11.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: osdmap e209: 8 total, 8 up, 8 in 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[40223]: pgmap v261: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: osdmap e209: 8 total, 8 up, 8 in 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:11 smithi049 ceph-mon[30017]: pgmap v261: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi049-75722-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: osdmap e209: 8 total, 8 up, 8 in 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:11 smithi129 ceph-mon[33364]: pgmap v261: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:46:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: osdmap e210: 8 total, 8 up, 8 in 2024-09-17T11:46:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: osdmap e210: 8 total, 8 up, 8 in 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:12.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:12.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:12.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: osdmap e210: 8 total, 8 up, 8 in 2024-09-17T11:46:12.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"} : dispatch 2024-09-17T11:46:12.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"}]': finished 2024-09-17T11:46:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[40223]: osdmap e211: 8 total, 8 up, 8 in 2024-09-17T11:46:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[40223]: pgmap v264: 228 pgs: 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"}]': finished 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[30017]: osdmap e211: 8 total, 8 up, 8 in 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:13 smithi049 ceph-mon[30017]: pgmap v264: 228 pgs: 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:13 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-27", "mode": "writeback"}]': finished 2024-09-17T11:46:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:13 smithi129 ceph-mon[33364]: osdmap e211: 8 total, 8 up, 8 in 2024-09-17T11:46:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:13 smithi129 ceph-mon[33364]: pgmap v264: 228 pgs: 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: osdmap e212: 8 total, 8 up, 8 in 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3433462328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: osdmap e212: 8 total, 8 up, 8 in 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3433462328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: osdmap e212: 8 total, 8 up, 8 in 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"} : dispatch 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3433462328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: osdmap e213: 8 total, 8 up, 8 in 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: pgmap v267: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: osdmap e214: 8 total, 8 up, 8 in 2024-09-17T11:46:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: osdmap e213: 8 total, 8 up, 8 in 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: pgmap v267: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: osdmap e214: 8 total, 8 up, 8 in 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4049293799' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi049-75722-36"}]': finished 2024-09-17T11:46:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi049-75736-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: osdmap e213: 8 total, 8 up, 8 in 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: pgmap v267: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-27"}]': finished 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: osdmap e214: 8 total, 8 up, 8 in 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:15.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[30017]: osdmap e215: 8 total, 8 up, 8 in 2024-09-17T11:46:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:17.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[30017]: pgmap v270: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[40223]: osdmap e215: 8 total, 8 up, 8 in 2024-09-17T11:46:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:17.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:17 smithi049 ceph-mon[40223]: pgmap v270: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:17.672 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:17 smithi129 ceph-mon[33364]: osdmap e215: 8 total, 8 up, 8 in 2024-09-17T11:46:17.673 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:17.673 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:17 smithi129 ceph-mon[33364]: pgmap v270: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:17.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: osdmap e216: 8 total, 8 up, 8 in 2024-09-17T11:46:18.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: osdmap e216: 8 total, 8 up, 8 in 2024-09-17T11:46:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi049-75722-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4230872007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi049-75736-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: osdmap e216: 8 total, 8 up, 8 in 2024-09-17T11:46:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:19.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:19.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: osdmap e217: 8 total, 8 up, 8 in 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[30017]: pgmap v273: 236 pgs: 40 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: osdmap e217: 8 total, 8 up, 8 in 2024-09-17T11:46:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:19 smithi049 ceph-mon[40223]: pgmap v273: 236 pgs: 40 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-17T11:46:19.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:19.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: osdmap e217: 8 total, 8 up, 8 in 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:19 smithi129 ceph-mon[33364]: pgmap v273: 236 pgs: 40 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-17T11:46:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: osdmap e218: 8 total, 8 up, 8 in 2024-09-17T11:46:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: osdmap e218: 8 total, 8 up, 8 in 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: osdmap e218: 8 total, 8 up, 8 in 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:20.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: osdmap e219: 8 total, 8 up, 8 in 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: pgmap v276: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: osdmap e220: 8 total, 8 up, 8 in 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: osdmap e219: 8 total, 8 up, 8 in 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: pgmap v276: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:21.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: osdmap e220: 8 total, 8 up, 8 in 2024-09-17T11:46:21.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2465384322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: osdmap e219: 8 total, 8 up, 8 in 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"} : dispatch 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: pgmap v276: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi049-75722-37"}]': finished 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: osdmap e220: 8 total, 8 up, 8 in 2024-09-17T11:46:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[30017]: osdmap e221: 8 total, 8 up, 8 in 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:22.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:22.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:22 smithi049 ceph-mon[40223]: osdmap e221: 8 total, 8 up, 8 in 2024-09-17T11:46:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi049-75722-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:22 smithi129 ceph-mon[33364]: osdmap e221: 8 total, 8 up, 8 in 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: pgmap v279: 236 pgs: 236 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-17T11:46:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"}]': finished 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: pgmap v279: 236 pgs: 236 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"}]': finished 2024-09-17T11:46:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"} : dispatch 2024-09-17T11:46:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: pgmap v279: 236 pgs: 236 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-17T11:46:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-29", "mode": "writeback"}]': finished 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: osdmap e222: 8 total, 8 up, 8 in 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: osdmap e223: 8 total, 8 up, 8 in 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: osdmap e222: 8 total, 8 up, 8 in 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: osdmap e223: 8 total, 8 up, 8 in 2024-09-17T11:46:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: osdmap e222: 8 total, 8 up, 8 in 2024-09-17T11:46:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi049-75722-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: osdmap e223: 8 total, 8 up, 8 in 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3185801308' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"} : dispatch 2024-09-17T11:46:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-17T11:46:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: osdmap e224: 8 total, 8 up, 8 in 2024-09-17T11:46:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: osdmap e224: 8 total, 8 up, 8 in 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-17T11:46:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-46"}]': finished 2024-09-17T11:46:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: osdmap e224: 8 total, 8 up, 8 in 2024-09-17T11:46:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: osdmap e225: 8 total, 8 up, 8 in 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: pgmap v285: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: osdmap e225: 8 total, 8 up, 8 in 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: pgmap v285: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:27.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi049-75736-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: osdmap e225: 8 total, 8 up, 8 in 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: pgmap v285: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]} : dispatch 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:28.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]}]': finished 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: osdmap e226: 8 total, 8 up, 8 in 2024-09-17T11:46:28.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]}]': finished 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: osdmap e226: 8 total, 8 up, 8 in 2024-09-17T11:46:28.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.14", "id": [3, 0]}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.3", "id": [1, 3]}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.6", "id": [6, 3]}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "253.1c", "id": [1, 3]}]': finished 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3699254576' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: osdmap e226: 8 total, 8 up, 8 in 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"} : dispatch 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"} : dispatch 2024-09-17T11:46:28.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T11:46:29.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: pgmap v288: 236 pgs: 2 peering, 10 unknown, 224 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: pgmap v288: 236 pgs: 2 peering, 10 unknown, 224 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:29.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:29 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:29.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi049-75736-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-29"}]': finished 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi049-75722-38"}]': finished 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: osdmap e227: 8 total, 8 up, 8 in 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: pgmap v288: 236 pgs: 2 peering, 10 unknown, 224 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:29 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:30.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:30.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[30017]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T11:46:30.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:30.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[40223]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T11:46:30.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi049-75722-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:30 smithi129 ceph-mon[33364]: osdmap e228: 8 total, 8 up, 8 in 2024-09-17T11:46:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T11:46:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: pgmap v291: 196 pgs: 1 unknown, 195 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:31.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: pgmap v291: 196 pgs: 1 unknown, 195 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T11:46:31.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:31.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: osdmap e229: 8 total, 8 up, 8 in 2024-09-17T11:46:31.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: pgmap v291: 196 pgs: 1 unknown, 195 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:46:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi049-75722-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: osdmap e230: 8 total, 8 up, 8 in 2024-09-17T11:46:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"} : dispatch 2024-09-17T11:46:32.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:32.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:32.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2660047861' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi049-75736-47"}]': finished 2024-09-17T11:46:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: osdmap e231: 8 total, 8 up, 8 in 2024-09-17T11:46:32.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:32.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: pgmap v294: 236 pgs: 236 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: pgmap v294: 236 pgs: 236 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:33.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: pgmap v294: 236 pgs: 236 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi049-75736-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: osdmap e232: 8 total, 8 up, 8 in 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:33.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:33.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:34.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:34.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:34.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T11:46:34.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:34.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:34.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:34.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: osdmap e233: 8 total, 8 up, 8 in 2024-09-17T11:46:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"} : dispatch 2024-09-17T11:46:34.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"} : dispatch 2024-09-17T11:46:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:35.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"}]': finished 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:35.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"}]': finished 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:35.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:35.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s; 16 B/s, 7 objects/s recovering 2024-09-17T11:46:35.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi049-75736-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1230237144' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi049-75722-39"}]': finished 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-31", "mode": "writeback"}]': finished 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: osdmap e234: 8 total, 8 up, 8 in 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:35.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:36.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:36.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:36.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:36.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi049-75722-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: osdmap e235: 8 total, 8 up, 8 in 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"} : dispatch 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:36.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:37.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:37.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:37.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: pgmap v300: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:37.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:37.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-31"}]': finished 2024-09-17T11:46:37.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: osdmap e236: 8 total, 8 up, 8 in 2024-09-17T11:46:37.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:37.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi049-75722-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: osdmap e237: 8 total, 8 up, 8 in 2024-09-17T11:46:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3866369953' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:38.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"} : dispatch 2024-09-17T11:46:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: pgmap v303: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T11:46:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: pgmap v303: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:39.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:39.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: pgmap v303: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi049-75736-48"}]': finished 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: osdmap e238: 8 total, 8 up, 8 in 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:39.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T11:46:41.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: pgmap v306: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:41.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: pgmap v306: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:41.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:41 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi049-75736-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: osdmap e239: 8 total, 8 up, 8 in 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: pgmap v306: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:41 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:46:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:42.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:42.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:42.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/593742074' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: osdmap e240: 8 total, 8 up, 8 in 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[40223]: pgmap v309: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:43.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:43 smithi049 ceph-mon[30017]: pgmap v309: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi049-75736-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi049-75722-40"}]': finished 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: osdmap e241: 8 total, 8 up, 8 in 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"} : dispatch 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:43 smithi129 ceph-mon[33364]: pgmap v309: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"}]': finished 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:44.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"}]': finished 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:44.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-33", "mode": "writeback"}]': finished 2024-09-17T11:46:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi049-75722-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: osdmap e242: 8 total, 8 up, 8 in 2024-09-17T11:46:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T11:46:45.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:45.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:45.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:45.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T11:46:45.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:45.655 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T11:46:45.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.656 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: osdmap e243: 8 total, 8 up, 8 in 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"} : dispatch 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: pgmap v312: 228 pgs: 228 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi049-75722-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-33"}]': finished 2024-09-17T11:46:45.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:45.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: osdmap e244: 8 total, 8 up, 8 in 2024-09-17T11:46:45.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3111510421' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:45.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"} : dispatch 2024-09-17T11:46:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T11:46:47.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[40223]: pgmap v315: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:47 smithi049 ceph-mon[30017]: pgmap v315: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:47.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi049-75736-49"}]': finished 2024-09-17T11:46:47.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: osdmap e245: 8 total, 8 up, 8 in 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:47.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-mon[33364]: pgmap v315: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:48.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:48.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:48.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi049-75736-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: osdmap e246: 8 total, 8 up, 8 in 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:48.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[30017]: pgmap v318: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:49 smithi049 ceph-mon[40223]: pgmap v318: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: osdmap e247: 8 total, 8 up, 8 in 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"} : dispatch 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:46:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:49 smithi129 ceph-mon[33364]: pgmap v318: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:50.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:50.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:50.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:50.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:50.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi049-75736-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/680745614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi049-75722-41"}]': finished 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: osdmap e248: 8 total, 8 up, 8 in 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"}]': finished 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:51.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:51.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"}]': finished 2024-09-17T11:46:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi049-75722-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: osdmap e249: 8 total, 8 up, 8 in 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"} : dispatch 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: pgmap v321: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:46:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-35", "mode": "writeback"}]': finished 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: osdmap e250: 8 total, 8 up, 8 in 2024-09-17T11:46:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi049-75722-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: osdmap e251: 8 total, 8 up, 8 in 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3756262540' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"} : dispatch 2024-09-17T11:46:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[40223]: pgmap v324: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:53.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[40223]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T11:46:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[30017]: pgmap v324: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:53 smithi049 ceph-mon[30017]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T11:46:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:53 smithi129 ceph-mon[33364]: pgmap v324: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi049-75736-50"}]': finished 2024-09-17T11:46:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:53 smithi129 ceph-mon[33364]: osdmap e252: 8 total, 8 up, 8 in 2024-09-17T11:46:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:54.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:54.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: osdmap e253: 8 total, 8 up, 8 in 2024-09-17T11:46:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:46:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: pgmap v327: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:55.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T11:46:55.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: pgmap v327: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T11:46:55.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.654 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: pgmap v327: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:46:55.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3728479082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi049-75736-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:46:55.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:55.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:55.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: osdmap e254: 8 total, 8 up, 8 in 2024-09-17T11:46:55.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3058233942' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:55.679 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"} : dispatch 2024-09-17T11:46:56.378 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7016 ms) 2024-09-17T11:46:56.378 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-09-17T11:46:56.378 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6968 ms) 2024-09-17T11:46:56.378 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-17T11:46:56.378 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6991 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7012 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7024 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7011 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7025 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7010 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6930 ms) 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-17T11:46:56.379 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7031 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7035 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6948 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6990 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7019 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7040 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7009 ms) 2024-09-17T11:46:56.380 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112059 ms total) 2024-09-17T11:46:56.381 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: 2024-09-17T11:46:56.381 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [----------] Global test environment tear-down 2024-09-17T11:46:56.381 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (189109 ms total) 2024-09-17T11:46:56.381 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio: [ PASSED ] 42 tests. 2024-09-17T11:46:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:46:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:46:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:46:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:57.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:57.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:57.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:57.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi049-75722-42"}]': finished 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: osdmap e255: 8 total, 8 up, 8 in 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:46:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:46:57.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:46:58.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:46:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:46:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:58.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T11:46:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:46:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:46:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:46:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: osdmap e256: 8 total, 8 up, 8 in 2024-09-17T11:46:58.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"} : dispatch 2024-09-17T11:46:58.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:46:58.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:46:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[30017]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T11:46:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[30017]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:46:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[40223]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T11:46:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:46:59 smithi049 ceph-mon[40223]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:46:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:59 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:46:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-35"}]': finished 2024-09-17T11:46:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:59 smithi129 ceph-mon[33364]: osdmap e257: 8 total, 8 up, 8 in 2024-09-17T11:46:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:46:59 smithi129 ceph-mon[33364]: pgmap v333: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:47:00.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:00.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:00 smithi049 ceph-mon[40223]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T11:47:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:00.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:00 smithi049 ceph-mon[30017]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T11:47:00.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi049-75736-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:00.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:00 smithi129 ceph-mon[33364]: osdmap e258: 8 total, 8 up, 8 in 2024-09-17T11:47:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T11:47:01.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:47:01.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:01.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:01 smithi129 ceph-mon[33364]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T11:47:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T11:47:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[40223]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: osdmap e259: 8 total, 8 up, 8 in 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:01 smithi049 ceph-mon[30017]: osdmap e260: 8 total, 8 up, 8 in 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: osdmap e261: 8 total, 8 up, 8 in 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1080852047' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"} : dispatch 2024-09-17T11:47:02.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:03.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:47:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T11:47:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:03.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:47:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi049-75736-52"}]': finished 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: osdmap e262: 8 total, 8 up, 8 in 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"} : dispatch 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"}]': finished 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:04.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:04.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"}]': finished 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-37", "mode": "writeback"}]': finished 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: osdmap e263: 8 total, 8 up, 8 in 2024-09-17T11:47:04.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:04.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:04.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[30017]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[30017]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[40223]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[40223]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T11:47:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:05 smithi129 ceph-mon[33364]: pgmap v342: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:05 smithi129 ceph-mon[33364]: osdmap e264: 8 total, 8 up, 8 in 2024-09-17T11:47:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"} : dispatch 2024-09-17T11:47:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[40223]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:06 smithi049 ceph-mon[30017]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T11:47:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:06 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi049-75736-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-37"}]': finished 2024-09-17T11:47:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:06 smithi129 ceph-mon[33364]: osdmap e265: 8 total, 8 up, 8 in 2024-09-17T11:47:07.728 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:07 smithi129 ceph-mon[33364]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:07.728 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:07 smithi129 ceph-mon[33364]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T11:47:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:07 smithi049 ceph-mon[30017]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:07 smithi049 ceph-mon[30017]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T11:47:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:07 smithi049 ceph-mon[40223]: pgmap v345: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:07 smithi049 ceph-mon[40223]: osdmap e266: 8 total, 8 up, 8 in 2024-09-17T11:47:08.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:08.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[30017]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T11:47:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:08.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:08.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:08.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[40223]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T11:47:08.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:08.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:08.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:08 smithi129 ceph-mon[33364]: osdmap e267: 8 total, 8 up, 8 in 2024-09-17T11:47:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:08.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: pgmap v348: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:09.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: pgmap v348: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:09.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: pgmap v348: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: osdmap e268: 8 total, 8 up, 8 in 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"} : dispatch 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:11.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[30017]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.653 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:11 smithi049 ceph-mon[40223]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:11.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3152040151' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi049-75736-53"}]': finished 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: osdmap e269: 8 total, 8 up, 8 in 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:11 smithi129 ceph-mon[33364]: pgmap v351: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:12.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:12.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi049-75736-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: osdmap e270: 8 total, 8 up, 8 in 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"} : dispatch 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"}]': finished 2024-09-17T11:47:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T11:47:13.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: pgmap v354: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"}]': finished 2024-09-17T11:47:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: pgmap v354: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:13.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:13.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-39", "mode": "writeback"}]': finished 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: osdmap e271: 8 total, 8 up, 8 in 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: pgmap v354: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi049-75736-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: osdmap e272: 8 total, 8 up, 8 in 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"} : dispatch 2024-09-17T11:47:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[40223]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T11:47:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:14.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:14 smithi049 ceph-mon[30017]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T11:47:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:14 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:14 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:47:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-39"}]': finished 2024-09-17T11:47:14.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:14 smithi129 ceph-mon[33364]: osdmap e273: 8 total, 8 up, 8 in 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: seed 76013 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (217 ms) 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10474 ms) 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (7 ms) 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10698 ms total) 2024-09-17T11:47:15.330 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7008 ms) 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8027 ms) 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9956 ms) 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: my_snaps [3] 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: my_snaps [4,3] 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: my_snaps [5,4,3] 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-09-17T11:47:15.331 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: promoting some heads 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: promoting from clones for snap 3 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: waiting for scrubs... 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: done waiting 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46059 ms) 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9962 ms) 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-17T11:47:15.332 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7993 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8028 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8012 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10026 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8934 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10046 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13988 ms) 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-17T11:47:15.333 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7978 ms) 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8026 ms) 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13054 ms) 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7923 ms) 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-17T11:47:15.334 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8019 ms) 2024-09-17T11:47:15.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:15 smithi049 ceph-mon[30017]: pgmap v357: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:15.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:15 smithi049 ceph-mon[30017]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T11:47:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:15 smithi049 ceph-mon[40223]: pgmap v357: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:15.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:15 smithi049 ceph-mon[40223]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T11:47:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:15 smithi129 ceph-mon[33364]: pgmap v357: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 119 op/s 2024-09-17T11:47:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:15 smithi129 ceph-mon[33364]: osdmap e274: 8 total, 8 up, 8 in 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: osdmap e275: 8 total, 8 up, 8 in 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2584962006' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"} : dispatch 2024-09-17T11:47:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: pgmap v360: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:17.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: pgmap v360: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:17.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:17.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: pgmap v360: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:17.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:17.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi049-75736-54"}]': finished 2024-09-17T11:47:17.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: osdmap e276: 8 total, 8 up, 8 in 2024-09-17T11:47:17.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:17.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:18.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T11:47:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:18.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi049-75736-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: osdmap e277: 8 total, 8 up, 8 in 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:18.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T11:47:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T11:47:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:19.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: pgmap v363: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:19.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: osdmap e278: 8 total, 8 up, 8 in 2024-09-17T11:47:19.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[30017]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:21 smithi049 ceph-mon[40223]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:21.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi049-75736-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: osdmap e279: 8 total, 8 up, 8 in 2024-09-17T11:47:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:47:21.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:21 smithi129 ceph-mon[33364]: pgmap v366: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:47:22.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:22 smithi049 ceph-mon[30017]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T11:47:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:47:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:22 smithi049 ceph-mon[40223]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T11:47:22.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:47:22.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:22 smithi129 ceph-mon[33364]: osdmap e280: 8 total, 8 up, 8 in 2024-09-17T11:47:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T11:47:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[30017]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:23 smithi049 ceph-mon[40223]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: osdmap e281: 8 total, 8 up, 8 in 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"} : dispatch 2024-09-17T11:47:23.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:23 smithi129 ceph-mon[33364]: pgmap v369: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"}]': finished 2024-09-17T11:47:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[40223]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T11:47:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:24.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"}]': finished 2024-09-17T11:47:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[30017]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T11:47:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:24 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-41"}]': finished 2024-09-17T11:47:24.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:24 smithi129 ceph-mon[33364]: osdmap e282: 8 total, 8 up, 8 in 2024-09-17T11:47:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"} : dispatch 2024-09-17T11:47:24.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:24 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:25.655 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:25.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T11:47:25.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.656 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.657 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.657 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[40223]: pgmap v372: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.659 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.659 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.659 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.659 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.659 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:25 smithi049 ceph-mon[30017]: pgmap v372: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2596378781' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi049-75736-55"}]': finished 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: osdmap e283: 8 total, 8 up, 8 in 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.829 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:25.830 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:25 smithi129 ceph-mon[33364]: pgmap v372: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:26.650 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:26.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi049-75736-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: osdmap e284: 8 total, 8 up, 8 in 2024-09-17T11:47:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:26.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-6","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-6","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:27.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:27.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-6","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: osdmap e285: 8 total, 8 up, 8 in 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi049-75736-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: osdmap e286: 8 total, 8 up, 8 in 2024-09-17T11:47:27.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:27.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-17T11:47:28.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"}]': finished 2024-09-17T11:47:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[40223]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T11:47:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:28.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:28.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:28.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"}]': finished 2024-09-17T11:47:28.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[30017]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T11:47:28.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:28.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_count","val": "8"}]': finished 2024-09-17T11:47:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:28 smithi129 ceph-mon[33364]: osdmap e287: 8 total, 8 up, 8 in 2024-09-17T11:47:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:28.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:29.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: pgmap v378: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:29.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:29.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: pgmap v378: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:29.652 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:29.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: pgmap v378: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: osdmap e288: 8 total, 8 up, 8 in 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:29.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.612 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: ok, hit_set contains 273:602f83fe:::foo:head 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9032 ms) 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg_num = 32 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 0 ls 1726573651,0 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 1 ls 1726573651,0 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 2 ls 1726573651,0 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 3 ls 1726573651,0 2024-09-17T11:47:30.613 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 4 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 5 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 6 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 7 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 8 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 9 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 10 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 11 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 12 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 13 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 14 ls 1726573651,0 2024-09-17T11:47:30.614 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 15 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 16 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 17 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 18 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 19 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 20 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 21 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 22 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 23 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 24 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 25 ls 1726573651,0 2024-09-17T11:47:30.615 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 26 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 27 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 28 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 29 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 30 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg 31 ls 1726573651,0 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: pg_num = 32 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6cac518f:::0:head 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:02547ec2:::1:head 2024-09-17T11:47:30.616 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f905c69b:::2:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:cfc208b3:::3:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d83876eb:::4:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b29083e3:::5:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c4fdafeb:::6:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:5c6b0b28:::7:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:bd63b0f1:::8:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e960b815:::9:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:52ea6a34:::10:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:89d3ae78:::11:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:de5d7c5f:::12:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:566253c9:::13:head 2024-09-17T11:47:30.617 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:62a1935d:::14:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:863748b0:::15:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3958e169:::16:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4d4dabf9:::17:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:8391935d:::18:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:28883081:::19:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:69259c59:::20:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4bdb80b7:::21:head 2024-09-17T11:47:30.618 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a11c5d71:::22:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:271af37b:::23:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:95b121be:::24:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:58d1031b:::25:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:0a050783:::26:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c709704c:::27:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:cbe56eaf:::28:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:86b4b162:::29:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:70d89383:::30:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:dd450c7c:::31:head 2024-09-17T11:47:30.619 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6d5729b1:::32:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c388f3fb:::33:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:56cfea31:::34:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9dbc1bf7:::35:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:40b74ccd:::36:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4d5aaf42:::37:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:920f362c:::38:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6cc53222:::39:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9cad833f:::40:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:1ea84d41:::41:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c4480ef6:::42:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a694361e:::43:head 2024-09-17T11:47:30.620 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d1bd33e9:::44:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ddc2cd5d:::45:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:2b782207:::46:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7b187fca:::47:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:90ecdf6f:::48:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a5ed95fe:::49:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ea0eaa55:::50:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f33ef17b:::51:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a0d1b2f6:::52:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:60c5229e:::53:head 2024-09-17T11:47:30.621 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:edcbc575:::54:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:102cf253:::55:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:efb7fb0b:::56:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:50d0a326:::57:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d4dc5daf:::58:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3a130462:::59:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ec87ed71:::60:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d5bc9454:::61:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3ddfe313:::62:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7c2816b9:::63:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:47e00e4d:::64:head 2024-09-17T11:47:30.622 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c6410c18:::65:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b48ed237:::66:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:cd63ad31:::67:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b179e92b:::68:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:0d9f741a:::69:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6d3352ae:::70:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c6d5c19e:::71:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:bc4729c3:::72:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:77e930b9:::73:head 2024-09-17T11:47:30.623 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:0abeecfd:::74:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b7c37e15:::75:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b6378398:::76:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:02bd68de:::77:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:cc795d2d:::78:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:630d4fea:::79:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e0d29ef5:::80:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:fd6f13d2:::81:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:606461d5:::82:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:eadbdc43:::83:head 2024-09-17T11:47:30.624 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:8761d0bb:::84:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9ef0186f:::85:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e0d41294:::86:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:961de695:::87:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:1423148f:::88:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:633a8fa2:::89:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a8653809:::90:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3dac8b33:::91:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:35aad435:::92:head 2024-09-17T11:47:30.625 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f6dcc343:::93:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:dbbdad87:::94:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:1cb48ce0:::95:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:03cd461c:::96:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:17a4ea99:::97:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9993c9a7:::98:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6394211c:::99:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:94c7ae57:::100:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6fdee5bb:::101:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9a477fd1:::102:head 2024-09-17T11:47:30.626 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:eb850916:::103:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:affc56b9:::104:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b42dc814:::105:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f319f8f0:::106:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9a40b9de:::107:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:8b524f28:::108:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e3de589f:::109:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:90f90a5b:::110:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a7b4f1d7:::111:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:af51766e:::112:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b6f90bd1:::113:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e0261208:::114:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c9569ef7:::115:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:61bebe50:::116:head 2024-09-17T11:47:30.627 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:fe93412b:::117:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d3d38bee:::118:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3100ba0c:::119:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d0560ada:::120:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f0ea8b35:::121:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:766f231a:::122:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a07a2582:::123:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:bd7c6b3a:::124:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:fb2ddaff:::125:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4408e1fe:::126:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ee1df7a7:::127:head 2024-09-17T11:47:30.628 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c3002909:::128:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4f48ffa9:::129:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:edf38733:::130:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c08425c0:::131:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:5f902d98:::132:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:41ea2c93:::133:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:813cee13:::134:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:0131818d:::135:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:26ba5a85:::136:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:381b8a5a:::137:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:28797e47:::138:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:bfca7f22:::139:head 2024-09-17T11:47:30.629 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:36807075:::140:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:80b03975:::141:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:5c15709b:::142:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f39ea15e:::143:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ea992956:::144:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:48887b1c:::145:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:9f24a9dd:::146:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:987f100b:::147:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d2dd3581:::148:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7fed1808:::149:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c80b70e9:::150:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:85ed90f9:::151:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:36428b24:::152:head 2024-09-17T11:47:30.630 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d044c34a:::153:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7c18bf58:::154:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d1c21232:::155:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a7a3c575:::156:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:87da0633:::157:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d5ac3822:::158:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3f20522d:::159:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6ca26563:::160:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:532ce135:::161:head 2024-09-17T11:47:30.631 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:c78863e6:::162:head 2024-09-17T11:47:30.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-17T11:47:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T11:47:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: osdmap e289: 8 total, 8 up, 8 in 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2090119630' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:30.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"} : dispatch 2024-09-17T11:47:31.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-43","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"}]': finished 2024-09-17T11:47:31.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:31 smithi129 ceph-mon[33364]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-43","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"}]': finished 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[40223]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: pgmap v381: 228 pgs: 228 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi049-76013-43","var": "pg_num","format": "json"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"} : dispatch 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi049-75736-56"}]': finished 2024-09-17T11:47:31.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-43"}]': finished 2024-09-17T11:47:31.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:31 smithi049 ceph-mon[30017]: osdmap e290: 8 total, 8 up, 8 in 2024-09-17T11:47:32.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:32.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:33.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:33.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[30017]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T11:47:33.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:33.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[30017]: pgmap v384: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:33.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:33.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[40223]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T11:47:33.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:33.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:33 smithi049 ceph-mon[40223]: pgmap v384: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:33.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi049-75736-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:33.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:33 smithi129 ceph-mon[33364]: osdmap e291: 8 total, 8 up, 8 in 2024-09-17T11:47:33.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:33.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:33 smithi129 ceph-mon[33364]: pgmap v384: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:34.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T11:47:34.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:34.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:34.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: osdmap e292: 8 total, 8 up, 8 in 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi049-75736-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: osdmap e293: 8 total, 8 up, 8 in 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:34.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:35.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[30017]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:35.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:35.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[30017]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[40223]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[40223]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T11:47:35.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:35.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:35.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:35 smithi129 ceph-mon[33364]: pgmap v387: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:35.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:35.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:35 smithi129 ceph-mon[33364]: osdmap e294: 8 total, 8 up, 8 in 2024-09-17T11:47:35.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:35.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:47:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:36.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:36.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:36.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:47:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: osdmap e295: 8 total, 8 up, 8 in 2024-09-17T11:47:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:47:36.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:37.755 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:37.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:37.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: osdmap e296: 8 total, 8 up, 8 in 2024-09-17T11:47:37.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"} : dispatch 2024-09-17T11:47:37.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:37.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:47:38.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:38.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:38.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:38.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:47:38.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:38.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:38.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2758702304' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi049-75736-57"}]': finished 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: osdmap e297: 8 total, 8 up, 8 in 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:38.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:39.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[30017]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:39.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:47:39.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:39.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[30017]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[40223]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[40223]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T11:47:39.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:39.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:39 smithi129 ceph-mon[33364]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:39.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:47:39.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi049-75736-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:39.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:39 smithi129 ceph-mon[33364]: osdmap e298: 8 total, 8 up, 8 in 2024-09-17T11:47:39.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:41 smithi049 ceph-mon[30017]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T11:47:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:41 smithi049 ceph-mon[30017]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:41 smithi049 ceph-mon[40223]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T11:47:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:41 smithi049 ceph-mon[40223]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:41.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:41 smithi129 ceph-mon[33364]: osdmap e299: 8 total, 8 up, 8 in 2024-09-17T11:47:41.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:41 smithi129 ceph-mon[33364]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:42.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:42.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T11:47:42.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:47:42.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:42.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:42.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:42.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi049-75736-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:42.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: osdmap e300: 8 total, 8 up, 8 in 2024-09-17T11:47:42.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:47:42.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:42.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:42.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[40223]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T11:47:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[40223]: pgmap v399: 236 pgs: 3 creating+activating, 233 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:47:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:47:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:43.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[30017]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T11:47:43.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[30017]: pgmap v399: 236 pgs: 3 creating+activating, 233 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:43.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:47:43.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:47:43.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:43 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:43 smithi129 ceph-mon[33364]: osdmap e301: 8 total, 8 up, 8 in 2024-09-17T11:47:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:43 smithi129 ceph-mon[33364]: pgmap v399: 236 pgs: 3 creating+activating, 233 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:47:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:47:43.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:43 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:47:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[30017]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T11:47:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[40223]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T11:47:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:44 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:44 smithi129 ceph-mon[33364]: osdmap e302: 8 total, 8 up, 8 in 2024-09-17T11:47:44.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[40223]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T11:47:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[40223]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[30017]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[30017]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:45 smithi129 ceph-mon[33364]: osdmap e303: 8 total, 8 up, 8 in 2024-09-17T11:47:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"} : dispatch 2024-09-17T11:47:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:45 smithi129 ceph-mon[33364]: pgmap v402: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:47:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/131914059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi049-75736-58"}]': finished 2024-09-17T11:47:46.802 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:46.803 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:46.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: osdmap e304: 8 total, 8 up, 8 in 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:46.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-mon[33364]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T11:47:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-mon[33364]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[40223]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T11:47:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[40223]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[30017]: osdmap e305: 8 total, 8 up, 8 in 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:47 smithi049 ceph-mon[30017]: pgmap v405: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:48.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:48.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:48 smithi049 ceph-mon[40223]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T11:47:48.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:48 smithi049 ceph-mon[30017]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T11:47:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:48.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:48 smithi129 ceph-mon[33364]: osdmap e306: 8 total, 8 up, 8 in 2024-09-17T11:47:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[40223]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T11:47:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[40223]: pgmap v408: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[30017]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T11:47:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[30017]: pgmap v408: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:49 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:49.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi049-75736-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:49 smithi129 ceph-mon[33364]: osdmap e307: 8 total, 8 up, 8 in 2024-09-17T11:47:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:49 smithi129 ceph-mon[33364]: pgmap v408: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:49 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:50.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:50 smithi049 ceph-mon[30017]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T11:47:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:50 smithi049 ceph-mon[40223]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T11:47:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:50 smithi129 ceph-mon[33364]: osdmap e308: 8 total, 8 up, 8 in 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:51.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[30017]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:51.909 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:51 smithi049 ceph-mon[40223]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T11:47:51.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: osdmap e309: 8 total, 8 up, 8 in 2024-09-17T11:47:51.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:51.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: pgmap v411: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:51.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:51.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:51 smithi129 ceph-mon[33364]: osdmap e310: 8 total, 8 up, 8 in 2024-09-17T11:47:52.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"}]': finished 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[30017]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:52.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"}]': finished 2024-09-17T11:47:52.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:52 smithi049 ceph-mon[40223]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T11:47:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1258333792' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"} : dispatch 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"} : dispatch 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi049-75736-59"}]': finished 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-45"}]': finished 2024-09-17T11:47:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:52 smithi129 ceph-mon[33364]: osdmap e311: 8 total, 8 up, 8 in 2024-09-17T11:47:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:53 smithi049 ceph-mon[30017]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:53 smithi049 ceph-mon[40223]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:53 smithi129 ceph-mon[33364]: pgmap v414: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:47:54.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[40223]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T11:47:54.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1957872366' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:54.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[30017]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T11:47:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1957872366' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:54 smithi129 ceph-mon[33364]: osdmap e312: 8 total, 8 up, 8 in 2024-09-17T11:47:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1957872366' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7462ddf6:::16.RoundTripAppendPP (3007 ms) 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3003 ms) 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3017 ms) 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3004 ms) 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-17T11:47:55.520 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: setting pool EIO 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: max_success 98, min_failed 99 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4007 ms) 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3005 ms) 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (118067 ms total) 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3013 ms) 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9017 ms) 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-17T11:47:55.521 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14955 ms) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3009 ms) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29994 ms total) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3020 ms) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3020 ms total) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13981 ms) 2024-09-17T11:47:55.522 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6981 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7035 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7021 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3006 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7026 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6920 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7018 ms) 2024-09-17T11:47:55.523 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7027 ms) 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7023 ms) 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7040 ms) 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7026 ms) 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7040 ms) 2024-09-17T11:47:55.524 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-09-17T11:47:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T11:47:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: pgmap v417: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[40223]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: pgmap v417: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:55 smithi049 ceph-mon[30017]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T11:47:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi049-75736-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: osdmap e313: 8 total, 8 up, 8 in 2024-09-17T11:47:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:47:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: pgmap v417: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:47:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:47:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:55 smithi129 ceph-mon[33364]: osdmap e314: 8 total, 8 up, 8 in 2024-09-17T11:47:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:47:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi049-75736-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:47:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: osdmap e315: 8 total, 8 up, 8 in 2024-09-17T11:47:56.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:56.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:47:56.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:47:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[30017]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:47:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[40223]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:47:57.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:57 smithi129 ceph-mon[33364]: pgmap v420: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:47:57.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:47:57.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:47:57.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:47:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:47:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T11:47:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"}]': finished 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"}]': finished 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:58.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:58.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: osdmap e316: 8 total, 8 up, 8 in 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"} : dispatch 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi049-75736-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-47", "mode": "writeback"}]': finished 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: osdmap e317: 8 total, 8 up, 8 in 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:58.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:47:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[40223]: pgmap v423: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:47:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[40223]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T11:47:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[30017]: pgmap v423: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:47:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[30017]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T11:47:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:47:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:59 smithi129 ceph-mon[33364]: pgmap v423: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:47:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:47:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:59 smithi129 ceph-mon[33364]: osdmap e318: 8 total, 8 up, 8 in 2024-09-17T11:47:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:47:59.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:47:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T11:48:01.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:01.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: osdmap e319: 8 total, 8 up, 8 in 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: osdmap e320: 8 total, 8 up, 8 in 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3715213333' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"} : dispatch 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi049-75736-61"}]': finished 2024-09-17T11:48:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:48:03.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:03.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T11:48:03.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:03.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:03.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: osdmap e321: 8 total, 8 up, 8 in 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: osdmap e322: 8 total, 8 up, 8 in 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:03.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:48:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[40223]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:48:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[40223]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T11:48:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[30017]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[30017]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:05 smithi129 ceph-mon[33364]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:48:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:05 smithi129 ceph-mon[33364]: osdmap e323: 8 total, 8 up, 8 in 2024-09-17T11:48:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[30017]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:48:06.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:06 smithi049 ceph-mon[40223]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T11:48:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi049-75736-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: osdmap e324: 8 total, 8 up, 8 in 2024-09-17T11:48:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"}]': finished 2024-09-17T11:48:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:06 smithi129 ceph-mon[33364]: osdmap e325: 8 total, 8 up, 8 in 2024-09-17T11:48:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: pgmap v434: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: pgmap v434: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T11:48:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:07.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:07.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: pgmap v434: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-47"} : dispatch 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: osdmap e326: 8 total, 8 up, 8 in 2024-09-17T11:48:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T11:48:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:08.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:08.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: osdmap e327: 8 total, 8 up, 8 in 2024-09-17T11:48:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"} : dispatch 2024-09-17T11:48:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[40223]: pgmap v437: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[40223]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[30017]: pgmap v437: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:09 smithi049 ceph-mon[30017]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T11:48:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:09 smithi129 ceph-mon[33364]: pgmap v437: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:09 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3868567980' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi049-75736-62"}]': finished 2024-09-17T11:48:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:09 smithi129 ceph-mon[33364]: osdmap e328: 8 total, 8 up, 8 in 2024-09-17T11:48:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:10.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi049-75736-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: osdmap e329: 8 total, 8 up, 8 in 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:11.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[40223]: pgmap v440: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:11.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[40223]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T11:48:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[30017]: pgmap v440: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:11.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:11.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:11.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[30017]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T11:48:11.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:11 smithi129 ceph-mon[33364]: pgmap v440: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:48:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:11 smithi129 ceph-mon[33364]: osdmap e330: 8 total, 8 up, 8 in 2024-09-17T11:48:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"} : dispatch 2024-09-17T11:48:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"}]': finished 2024-09-17T11:48:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[40223]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T11:48:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"}]': finished 2024-09-17T11:48:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:12 smithi049 ceph-mon[30017]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T11:48:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:12 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi049-75736-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-49", "mode": "readproxy"}]': finished 2024-09-17T11:48:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:12 smithi129 ceph-mon[33364]: osdmap e331: 8 total, 8 up, 8 in 2024-09-17T11:48:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:13 smithi049 ceph-mon[30017]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:13 smithi049 ceph-mon[30017]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T11:48:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:13 smithi049 ceph-mon[40223]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:13 smithi049 ceph-mon[40223]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T11:48:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:13 smithi129 ceph-mon[33364]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:13 smithi129 ceph-mon[33364]: osdmap e332: 8 total, 8 up, 8 in 2024-09-17T11:48:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[30017]: pgmap v446: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[30017]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T11:48:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[40223]: pgmap v446: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[40223]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T11:48:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:15 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:15 smithi129 ceph-mon[33364]: pgmap v446: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:48:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:15 smithi129 ceph-mon[33364]: osdmap e333: 8 total, 8 up, 8 in 2024-09-17T11:48:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:15 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[40223]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T11:48:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[30017]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T11:48:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:16.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:16 smithi129 ceph-mon[33364]: osdmap e334: 8 total, 8 up, 8 in 2024-09-17T11:48:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"} : dispatch 2024-09-17T11:48:16.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1400926248' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi049-75736-63"}]': finished 2024-09-17T11:48:17.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[30017]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:17.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[30017]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T11:48:17.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[40223]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[40223]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:17.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-mon[33364]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-mon[33364]: osdmap e335: 8 total, 8 up, 8 in 2024-09-17T11:48:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[40223]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T11:48:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[30017]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T11:48:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi049-75736-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:18 smithi129 ceph-mon[33364]: osdmap e336: 8 total, 8 up, 8 in 2024-09-17T11:48:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:19 smithi049 ceph-mon[40223]: pgmap v452: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:19 smithi049 ceph-mon[40223]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T11:48:19.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:19 smithi049 ceph-mon[30017]: pgmap v452: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:19.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:19 smithi049 ceph-mon[30017]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T11:48:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:19 smithi129 ceph-mon[33364]: pgmap v452: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:19 smithi129 ceph-mon[33364]: osdmap e337: 8 total, 8 up, 8 in 2024-09-17T11:48:20.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[40223]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T11:48:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[40223]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T11:48:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[30017]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T11:48:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:20 smithi049 ceph-mon[30017]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T11:48:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi049-75736-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:20 smithi129 ceph-mon[33364]: osdmap e338: 8 total, 8 up, 8 in 2024-09-17T11:48:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:20 smithi129 ceph-mon[33364]: osdmap e339: 8 total, 8 up, 8 in 2024-09-17T11:48:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:21 smithi049 ceph-mon[40223]: pgmap v455: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:21 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:21 smithi049 ceph-mon[30017]: pgmap v455: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:21 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:21 smithi129 ceph-mon[33364]: pgmap v455: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:48:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:21 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:22 smithi049 ceph-mon[40223]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T11:48:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:22.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:22 smithi049 ceph-mon[30017]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T11:48:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:22.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:22 smithi129 ceph-mon[33364]: osdmap e340: 8 total, 8 up, 8 in 2024-09-17T11:48:22.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:23.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:23.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T11:48:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:23.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: osdmap e341: 8 total, 8 up, 8 in 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"} : dispatch 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4207335918' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi049-75736-64"}]': finished 2024-09-17T11:48:24.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:24.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[30017]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:24.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:24.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:24 smithi049 ceph-mon[40223]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T11:48:24.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:24.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: osdmap e342: 8 total, 8 up, 8 in 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:24.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"}]': finished 2024-09-17T11:48:24.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi049-75736-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:24.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:24 smithi129 ceph-mon[33364]: osdmap e343: 8 total, 8 up, 8 in 2024-09-17T11:48:25.649 INFO:tasks.workunit.client.0.smithi049.stdout: 3:head 2024-09-17T11:48:25.649 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:5d165639:::164:head 2024-09-17T11:48:25.649 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f43765fc:::165:head 2024-09-17T11:48:25.649 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b4c720e9:::166:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e694b040:::167:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:afa38db2:::168:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:77ba9f53:::169:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:87495034:::170:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7c96bf0e:::171:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:dbe346cc:::172:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:e943ec24:::173:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f97a9c0c:::174:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:6f26e74d:::175:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4f95e106:::176:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:0e6f2f8f:::177:head 2024-09-17T11:48:25.650 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:05db05f1:::178:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:38a78d66:::179:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:d095610b:::180:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:a1a9d709:::181:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:1e5d39db:::182:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:f7df4fb9:::183:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:03a7f161:::184:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ba70721e:::185:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:28e5662d:::186:head 2024-09-17T11:48:25.651 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:973d52de:::187:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:4303eb1c:::188:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:b990b48e:::189:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:29b8165b:::190:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:3547f197:::191:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:7e260936:::192:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:1abec7b1:::193:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:10fdda93:::194:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:15817eea:::195:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:770bab57:::196:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:ed9e13e7:::197:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:71471a8f:::198:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: checking for 275:10fb1d02:::199:head 2024-09-17T11:48:25.652 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8052 ms) 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: first is 1726573660 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,1726573668,1726573669,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,1726573668,1726573669,0 2024-09-17T11:48:25.653 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573660,1726573662,1726573663,1726573665,1726573666,1726573668,1726573669,0 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726573663,1726573665,1726573666,1726573668,1726573669,1726573671,1726573672,0 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: first now 1726573663, trimmed 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21099 ms) 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: foo0 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14057 ms) 2024-09-17T11:48:25.654 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-49"} : dispatch 2024-09-17T11:48:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[40223]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[40223]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[30017]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[30017]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T11:48:26.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:26.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:26.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:26 smithi129 ceph-mon[33364]: osdmap e344: 8 total, 8 up, 8 in 2024-09-17T11:48:26.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi049-75736-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:26.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:26 smithi129 ceph-mon[33364]: osdmap e345: 8 total, 8 up, 8 in 2024-09-17T11:48:26.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:26.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:27.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:27.965 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:27 smithi129 ceph-mon[33364]: pgmap v464: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:27.965 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:27.965 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:27.965 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:27 smithi129 ceph-mon[33364]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T11:48:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[40223]: pgmap v464: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[40223]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T11:48:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[30017]: pgmap v464: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:27 smithi049 ceph-mon[30017]: osdmap e346: 8 total, 8 up, 8 in 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:28.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:28.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:48:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: osdmap e347: 8 total, 8 up, 8 in 2024-09-17T11:48:29.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:29.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:29.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: pgmap v467: 236 pgs: 40 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:29.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: pgmap v467: 236 pgs: 40 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: pgmap v467: 236 pgs: 40 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: osdmap e348: 8 total, 8 up, 8 in 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3408489863' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"} : dispatch 2024-09-17T11:48:30.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"} : dispatch 2024-09-17T11:48:30.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:30 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:30.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"}]': finished 2024-09-17T11:48:30.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:30.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:30 smithi129 ceph-mon[33364]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T11:48:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"}]': finished 2024-09-17T11:48:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:31.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[40223]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T11:48:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:48:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-51", "mode": "writeback"}]': finished 2024-09-17T11:48:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi049-75736-65"}]': finished 2024-09-17T11:48:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:30 smithi049 ceph-mon[30017]: osdmap e349: 8 total, 8 up, 8 in 2024-09-17T11:48:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: pgmap v470: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: pgmap v470: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:32.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: pgmap v470: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:48:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:48:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T11:48:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi049-75736-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:48:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:48:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: osdmap e350: 8 total, 8 up, 8 in 2024-09-17T11:48:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:48:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[30017]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[30017]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T11:48:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[40223]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[40223]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T11:48:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:33 smithi129 ceph-mon[33364]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:48:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:33 smithi129 ceph-mon[33364]: osdmap e351: 8 total, 8 up, 8 in 2024-09-17T11:48:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:48:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T11:48:35.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:34 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:48:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi049-75736-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:48:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: osdmap e352: 8 total, 8 up, 8 in 2024-09-17T11:48:35.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:48:35.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:34 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[30017]: pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[30017]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[40223]: pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[40223]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T11:48:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:35 smithi129 ceph-mon[33364]: pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-17T11:48:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:48:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:35 smithi129 ceph-mon[33364]: osdmap e353: 8 total, 8 up, 8 in 2024-09-17T11:48:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:48:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[30017]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[40223]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-51","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:48:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:36 smithi129 ceph-mon[33364]: osdmap e354: 8 total, 8 up, 8 in 2024-09-17T11:48:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:37.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3026 ms) 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7033 ms) 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7025 ms) 2024-09-17T11:48:37.731 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7012 ms) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7049 ms) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7012 ms) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7076 ms) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139378 ms total) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (290461 ms total) 2024-09-17T11:48:37.732 INFO:tasks.workunit.client.0.smithi049.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-17T11:48:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:48:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T11:48:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:38.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:48:38.180 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:38.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3439137291' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:38.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: osdmap e355: 8 total, 8 up, 8 in 2024-09-17T11:48:38.181 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"} : dispatch 2024-09-17T11:48:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[40223]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T11:48:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[30017]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T11:48:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi049-75736-66"}]': finished 2024-09-17T11:48:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:38 smithi129 ceph-mon[33364]: osdmap e356: 8 total, 8 up, 8 in 2024-09-17T11:48:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:39 smithi049 ceph-mon[40223]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:39 smithi049 ceph-mon[30017]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:39 smithi129 ceph-mon[33364]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:41 smithi049 ceph-mon[40223]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:42.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:41 smithi049 ceph-mon[30017]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:41 smithi129 ceph-mon[33364]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[40223]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:43.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:48:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[30017]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:43.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:48:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:43 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:43 smithi129 ceph-mon[33364]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:48:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:44 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:44 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:44 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:48:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:44 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:48:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:44 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:48:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:48:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:44 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:45.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:45 smithi129 ceph-mon[33364]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:45 smithi049 ceph-mon[40223]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:45.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:45 smithi049 ceph-mon[30017]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:48:46.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:46.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:46.801 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:46.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:46.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:46.802 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:46.802 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:47.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:47.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-mon[33364]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T11:48:47.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-mon[33364]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-09-17T11:48:47.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:47.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[30017]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[30017]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[40223]: osdmap e357: 8 total, 8 up, 8 in 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:47 smithi049 ceph-mon[40223]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-09-17T11:48:48.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T11:48:48.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:48.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"}]': finished 2024-09-17T11:48:48.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: osdmap e358: 8 total, 8 up, 8 in 2024-09-17T11:48:48.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-51"} : dispatch 2024-09-17T11:48:48.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:49 smithi049 ceph-mon[40223]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T11:48:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:49 smithi049 ceph-mon[40223]: pgmap v490: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:48:49.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:49 smithi049 ceph-mon[30017]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T11:48:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:49 smithi049 ceph-mon[30017]: pgmap v490: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:48:49.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:49 smithi129 ceph-mon[33364]: osdmap e359: 8 total, 8 up, 8 in 2024-09-17T11:48:49.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:49 smithi129 ceph-mon[33364]: pgmap v490: 196 pgs: 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:48:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:48:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[40223]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T11:48:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:50.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:48:50.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[30017]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T11:48:50.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:50.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:50 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:48:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:50 smithi129 ceph-mon[33364]: osdmap e360: 8 total, 8 up, 8 in 2024-09-17T11:48:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:50.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:51.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:51.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T11:48:51.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: pgmap v493: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: pgmap v493: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:51.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: osdmap e361: 8 total, 8 up, 8 in 2024-09-17T11:48:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: pgmap v493: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:48:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-53"} : dispatch 2024-09-17T11:48:52.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:52 smithi049 ceph-mon[30017]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T11:48:52.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:52 smithi049 ceph-mon[40223]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T11:48:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:52 smithi129 ceph-mon[33364]: osdmap e362: 8 total, 8 up, 8 in 2024-09-17T11:48:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[30017]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T11:48:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[30017]: pgmap v496: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:53.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[40223]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T11:48:53.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:53 smithi049 ceph-mon[40223]: pgmap v496: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:53 smithi129 ceph-mon[33364]: osdmap e363: 8 total, 8 up, 8 in 2024-09-17T11:48:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:53 smithi129 ceph-mon[33364]: pgmap v496: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:54.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:48:54.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:54.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: osdmap e364: 8 total, 8 up, 8 in 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-55"} : dispatch 2024-09-17T11:48:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:55 smithi049 ceph-mon[40223]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T11:48:55.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:55 smithi049 ceph-mon[40223]: pgmap v499: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:55 smithi049 ceph-mon[30017]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T11:48:55.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:55 smithi049 ceph-mon[30017]: pgmap v499: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:55 smithi129 ceph-mon[33364]: osdmap e365: 8 total, 8 up, 8 in 2024-09-17T11:48:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:55 smithi129 ceph-mon[33364]: pgmap v499: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:48:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:48:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:48:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[30017]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T11:48:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[40223]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T11:48:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:56.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:56 smithi129 ceph-mon[33364]: osdmap e366: 8 total, 8 up, 8 in 2024-09-17T11:48:56.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:57.827 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: pgmap v502: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:57.828 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T11:48:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: pgmap v502: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: osdmap e367: 8 total, 8 up, 8 in 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:48:57.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-57"} : dispatch 2024-09-17T11:48:57.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: pgmap v502: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:48:57.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:48:57.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:48:58.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:48:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:48:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:58 smithi049 ceph-mon[40223]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T11:48:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:58.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:58 smithi049 ceph-mon[30017]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T11:48:58.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:58.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:58 smithi129 ceph-mon[33364]: osdmap e368: 8 total, 8 up, 8 in 2024-09-17T11:48:58.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:48:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[40223]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T11:48:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[40223]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[30017]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[30017]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:48:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:48:59 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:48:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:59 smithi129 ceph-mon[33364]: osdmap e369: 8 total, 8 up, 8 in 2024-09-17T11:48:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:48:59.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:59 smithi129 ceph-mon[33364]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:48:59.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:48:59 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T11:49:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: osdmap e370: 8 total, 8 up, 8 in 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-59"} : dispatch 2024-09-17T11:49:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:01 smithi049 ceph-mon[40223]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T11:49:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:01 smithi049 ceph-mon[40223]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:49:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:01 smithi049 ceph-mon[30017]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T11:49:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:01 smithi049 ceph-mon[30017]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:49:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:01 smithi129 ceph-mon[33364]: osdmap e371: 8 total, 8 up, 8 in 2024-09-17T11:49:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:01 smithi129 ceph-mon[33364]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-17T11:49:02.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[40223]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T11:49:02.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[30017]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T11:49:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:02 smithi129 ceph-mon[33364]: osdmap e372: 8 total, 8 up, 8 in 2024-09-17T11:49:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:03.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T11:49:03.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:03.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:49:03.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:03.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:03.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: osdmap e373: 8 total, 8 up, 8 in 2024-09-17T11:49:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:49:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:04.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: osdmap e374: 8 total, 8 up, 8 in 2024-09-17T11:49:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-61"} : dispatch 2024-09-17T11:49:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:05 smithi049 ceph-mon[30017]: pgmap v513: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 762 B/s wr, 2 op/s 2024-09-17T11:49:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:05 smithi049 ceph-mon[30017]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T11:49:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:05 smithi049 ceph-mon[40223]: pgmap v513: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 762 B/s wr, 2 op/s 2024-09-17T11:49:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:05 smithi049 ceph-mon[40223]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T11:49:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:05 smithi129 ceph-mon[33364]: pgmap v513: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 762 B/s wr, 2 op/s 2024-09-17T11:49:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:05 smithi129 ceph-mon[33364]: osdmap e375: 8 total, 8 up, 8 in 2024-09-17T11:49:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[40223]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T11:49:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[40223]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[30017]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:06 smithi049 ceph-mon[30017]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T11:49:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:06 smithi129 ceph-mon[33364]: osdmap e376: 8 total, 8 up, 8 in 2024-09-17T11:49:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:06.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:06 smithi129 ceph-mon[33364]: osdmap e377: 8 total, 8 up, 8 in 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[30017]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:07 smithi049 ceph-mon[40223]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T11:49:07.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: pgmap v516: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:07 smithi129 ceph-mon[33364]: osdmap e378: 8 total, 8 up, 8 in 2024-09-17T11:49:08.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:08.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:09 smithi049 ceph-mon[40223]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:09 smithi049 ceph-mon[40223]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T11:49:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:09 smithi049 ceph-mon[30017]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:09 smithi049 ceph-mon[30017]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T11:49:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:09 smithi129 ceph-mon[33364]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:09 smithi129 ceph-mon[33364]: osdmap e379: 8 total, 8 up, 8 in 2024-09-17T11:49:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:10 smithi049 ceph-mon[40223]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T11:49:10.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:10 smithi049 ceph-mon[30017]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T11:49:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:10 smithi129 ceph-mon[33364]: osdmap e380: 8 total, 8 up, 8 in 2024-09-17T11:49:11.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:11 smithi049 ceph-mon[40223]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:11.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:11 smithi049 ceph-mon[40223]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T11:49:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:11 smithi049 ceph-mon[30017]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:11 smithi049 ceph-mon[30017]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T11:49:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:11 smithi129 ceph-mon[33364]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:49:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:11 smithi129 ceph-mon[33364]: osdmap e381: 8 total, 8 up, 8 in 2024-09-17T11:49:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:12 smithi049 ceph-mon[40223]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T11:49:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:12.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:12 smithi049 ceph-mon[30017]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T11:49:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:12 smithi129 ceph-mon[33364]: osdmap e382: 8 total, 8 up, 8 in 2024-09-17T11:49:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:13.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:13 smithi049 ceph-mon[40223]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-17T11:49:13.903 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:13 smithi049 ceph-mon[30017]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-17T11:49:13.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:13 smithi129 ceph-mon[33364]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-17T11:49:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:14 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:14.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:14 smithi049 ceph-mon[30017]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T11:49:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:14 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:14 smithi049 ceph-mon[40223]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T11:49:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:14 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:49:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:14 smithi129 ceph-mon[33364]: osdmap e383: 8 total, 8 up, 8 in 2024-09-17T11:49:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:15 smithi049 ceph-mon[30017]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-17T11:49:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:15 smithi049 ceph-mon[40223]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-17T11:49:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:15 smithi129 ceph-mon[33364]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-17T11:49:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:17.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:17 smithi049 ceph-mon[40223]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-17T11:49:17.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:17 smithi049 ceph-mon[30017]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-17T11:49:17.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:17 smithi129 ceph-mon[33364]: pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-17T11:49:17.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:18.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:19 smithi049 ceph-mon[40223]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-17T11:49:19.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:19 smithi049 ceph-mon[30017]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-17T11:49:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:19 smithi129 ceph-mon[33364]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-17T11:49:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:21 smithi129 ceph-mon[33364]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:49:22.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:21 smithi049 ceph-mon[40223]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:49:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:21 smithi049 ceph-mon[30017]: pgmap v530: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:49:22.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:22 smithi129 ceph-mon[33364]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T11:49:23.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:22 smithi049 ceph-mon[30017]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T11:49:23.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:22 smithi049 ceph-mon[40223]: osdmap e384: 8 total, 8 up, 8 in 2024-09-17T11:49:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:23 smithi129 ceph-mon[33364]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:23 smithi049 ceph-mon[40223]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:24.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:23 smithi049 ceph-mon[30017]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:25 smithi049 ceph-mon[40223]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T11:49:25.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:25 smithi049 ceph-mon[40223]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:25.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:25 smithi049 ceph-mon[30017]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T11:49:25.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:25 smithi049 ceph-mon[30017]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:25.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:25 smithi129 ceph-mon[33364]: osdmap e385: 8 total, 8 up, 8 in 2024-09-17T11:49:25.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:25 smithi129 ceph-mon[33364]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:27 smithi049 ceph-mon[30017]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:27 smithi049 ceph-mon[40223]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:27 smithi129 ceph-mon[33364]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:27.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:28.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:29 smithi049 ceph-mon[40223]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:29.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:29 smithi049 ceph-mon[30017]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:29 smithi129 ceph-mon[33364]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:31.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:31 smithi049 ceph-mon[40223]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:31.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:31 smithi049 ceph-mon[30017]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:31.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:31 smithi129 ceph-mon[33364]: pgmap v537: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:49:33.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:33 smithi049 ceph-mon[40223]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:33.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:33 smithi049 ceph-mon[40223]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T11:49:33.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:33 smithi049 ceph-mon[30017]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:33.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:33 smithi049 ceph-mon[30017]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T11:49:33.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:33 smithi129 ceph-mon[33364]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:33.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:33 smithi129 ceph-mon[33364]: osdmap e386: 8 total, 8 up, 8 in 2024-09-17T11:49:35.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:35 smithi049 ceph-mon[40223]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:35.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:35 smithi049 ceph-mon[30017]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:35.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:35 smithi129 ceph-mon[33364]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:37.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:37 smithi049 ceph-mon[30017]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:37.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:37 smithi049 ceph-mon[40223]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:37 smithi129 ceph-mon[33364]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:37.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:38.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:38.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:38.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:39.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:39 smithi129 ceph-mon[33364]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:39.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:39 smithi129 ceph-mon[33364]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T11:49:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:39 smithi049 ceph-mon[40223]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:39 smithi049 ceph-mon[40223]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T11:49:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:39 smithi049 ceph-mon[30017]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:49:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:39 smithi049 ceph-mon[30017]: osdmap e387: 8 total, 8 up, 8 in 2024-09-17T11:49:41.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:41 smithi129 ceph-mon[33364]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:49:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:41 smithi049 ceph-mon[40223]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:49:42.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:41 smithi049 ceph-mon[30017]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:49:42.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:43.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:43 smithi129 ceph-mon[33364]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T11:49:43.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:43.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:43.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:43.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[40223]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T11:49:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[30017]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-63"} : dispatch 2024-09-17T11:49:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[40223]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T11:49:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:49:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:49:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:49:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:49:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[30017]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T11:49:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:49:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:49:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:49:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:44 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:49:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:44 smithi129 ceph-mon[33364]: osdmap e388: 8 total, 8 up, 8 in 2024-09-17T11:49:45.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:44 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:49:45.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:44 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:49:45.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:44 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:49:45.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:44 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:49:46.051 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: pgmap v547: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[40223]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: pgmap v547: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T11:49:46.052 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.053 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.053 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:46.053 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:45 smithi049 ceph-mon[30017]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: pgmap v547: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: osdmap e389: 8 total, 8 up, 8 in 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:49:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:45 smithi129 ceph-mon[33364]: osdmap e390: 8 total, 8 up, 8 in 2024-09-17T11:49:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:47.078 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:49:47.078 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:47.078 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:49:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[30017]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[30017]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T11:49:48.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[40223]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:47 smithi049 ceph-mon[40223]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T11:49:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:47 smithi129 ceph-mon[33364]: pgmap v550: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:49:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:49:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:47 smithi129 ceph-mon[33364]: osdmap e391: 8 total, 8 up, 8 in 2024-09-17T11:49:48.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:48 smithi049 ceph-mon[40223]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T11:49:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:48 smithi049 ceph-mon[30017]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T11:49:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:48 smithi129 ceph-mon[33364]: osdmap e392: 8 total, 8 up, 8 in 2024-09-17T11:49:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:49 smithi049 ceph-mon[30017]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:49 smithi049 ceph-mon[30017]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T11:49:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:49 smithi049 ceph-mon[40223]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:49 smithi049 ceph-mon[40223]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T11:49:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:49 smithi129 ceph-mon[33364]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:49 smithi129 ceph-mon[33364]: osdmap e393: 8 total, 8 up, 8 in 2024-09-17T11:49:51.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:50 smithi049 ceph-mon[40223]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T11:49:51.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:50 smithi049 ceph-mon[30017]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T11:49:51.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:50 smithi129 ceph-mon[33364]: osdmap e394: 8 total, 8 up, 8 in 2024-09-17T11:49:52.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:51 smithi049 ceph-mon[40223]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:51 smithi049 ceph-mon[30017]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:51 smithi129 ceph-mon[33364]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:49:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:53 smithi049 ceph-mon[30017]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T11:49:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:53 smithi049 ceph-mon[40223]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T11:49:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:53 smithi129 ceph-mon[33364]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T11:49:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:55 smithi049 ceph-mon[30017]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T11:49:55.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:55 smithi049 ceph-mon[30017]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-17T11:49:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:55 smithi049 ceph-mon[40223]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T11:49:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:55 smithi049 ceph-mon[40223]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-17T11:49:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:55 smithi129 ceph-mon[33364]: osdmap e395: 8 total, 8 up, 8 in 2024-09-17T11:49:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:55 smithi129 ceph-mon[33364]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-17T11:49:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:49:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:49:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:49:57.883 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:57 smithi129 ceph-mon[33364]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-17T11:49:57.883 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:57 smithi049 ceph-mon[40223]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-17T11:49:57.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:57 smithi049 ceph-mon[30017]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 520 B/s wr, 3 op/s 2024-09-17T11:49:57.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:49:58.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:49:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:49:58.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:58.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:58.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:49:59.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:49:59 smithi049 ceph-mon[30017]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:49:59 smithi049 ceph-mon[40223]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-17T11:49:59.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:49:59 smithi129 ceph-mon[33364]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 461 B/s wr, 3 op/s 2024-09-17T11:50:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[30017]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:00 smithi049 ceph-mon[40223]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T11:50:00.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-65"} : dispatch 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T11:50:00.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:00 smithi129 ceph-mon[33364]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T11:50:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:01 smithi049 ceph-mon[30017]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-17T11:50:01.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:01 smithi049 ceph-mon[30017]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T11:50:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:01 smithi049 ceph-mon[40223]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-17T11:50:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:01 smithi049 ceph-mon[40223]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T11:50:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:01 smithi129 ceph-mon[33364]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-17T11:50:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:01 smithi129 ceph-mon[33364]: osdmap e396: 8 total, 8 up, 8 in 2024-09-17T11:50:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:02 smithi129 ceph-mon[33364]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T11:50:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:03.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[40223]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T11:50:03.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:03.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:03.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[30017]: osdmap e397: 8 total, 8 up, 8 in 2024-09-17T11:50:03.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:03.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:03.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:03 smithi129 ceph-mon[33364]: pgmap v565: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:03 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:03 smithi129 ceph-mon[33364]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T11:50:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:04.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[40223]: pgmap v565: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:04.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:04.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:04.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[40223]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T11:50:04.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:04.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[30017]: pgmap v565: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:04.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[30017]: osdmap e398: 8 total, 8 up, 8 in 2024-09-17T11:50:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[30017]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T11:50:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[40223]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T11:50:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:05.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:04 smithi129 ceph-mon[33364]: osdmap e399: 8 total, 8 up, 8 in 2024-09-17T11:50:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-67"} : dispatch 2024-09-17T11:50:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[30017]: pgmap v568: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[30017]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T11:50:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[30017]: osdmap e401: 8 total, 8 up, 8 in 2024-09-17T11:50:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[40223]: pgmap v568: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[40223]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T11:50:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[40223]: osdmap e401: 8 total, 8 up, 8 in 2024-09-17T11:50:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:05 smithi129 ceph-mon[33364]: pgmap v568: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:05 smithi129 ceph-mon[33364]: osdmap e400: 8 total, 8 up, 8 in 2024-09-17T11:50:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:05 smithi129 ceph-mon[33364]: osdmap e401: 8 total, 8 up, 8 in 2024-09-17T11:50:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: pgmap v571: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: osdmap e402: 8 total, 8 up, 8 in 2024-09-17T11:50:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: pgmap v571: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: osdmap e402: 8 total, 8 up, 8 in 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:08.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:08.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: pgmap v571: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: osdmap e402: 8 total, 8 up, 8 in 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:08.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-69"} : dispatch 2024-09-17T11:50:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:08 smithi049 ceph-mon[40223]: osdmap e403: 8 total, 8 up, 8 in 2024-09-17T11:50:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:08 smithi049 ceph-mon[30017]: osdmap e403: 8 total, 8 up, 8 in 2024-09-17T11:50:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:09.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:08 smithi129 ceph-mon[33364]: osdmap e403: 8 total, 8 up, 8 in 2024-09-17T11:50:09.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[40223]: pgmap v574: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[40223]: osdmap e404: 8 total, 8 up, 8 in 2024-09-17T11:50:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:10.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[30017]: pgmap v574: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[30017]: osdmap e404: 8 total, 8 up, 8 in 2024-09-17T11:50:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:09 smithi129 ceph-mon[33364]: pgmap v574: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:09 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:09 smithi129 ceph-mon[33364]: osdmap e404: 8 total, 8 up, 8 in 2024-09-17T11:50:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:11.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:11.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[40223]: osdmap e405: 8 total, 8 up, 8 in 2024-09-17T11:50:11.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:11.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[30017]: osdmap e405: 8 total, 8 up, 8 in 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:11.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:11.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:10 smithi129 ceph-mon[33364]: osdmap e405: 8 total, 8 up, 8 in 2024-09-17T11:50:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:50:12.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[40223]: pgmap v577: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:12.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:50:12.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[40223]: osdmap e406: 8 total, 8 up, 8 in 2024-09-17T11:50:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[30017]: pgmap v577: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:50:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:11 smithi049 ceph-mon[30017]: osdmap e406: 8 total, 8 up, 8 in 2024-09-17T11:50:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:11 smithi129 ceph-mon[33364]: pgmap v577: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:50:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:50:12.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:11 smithi129 ceph-mon[33364]: osdmap e406: 8 total, 8 up, 8 in 2024-09-17T11:50:13.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:12 smithi049 ceph-mon[40223]: osdmap e407: 8 total, 8 up, 8 in 2024-09-17T11:50:13.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:12 smithi049 ceph-mon[30017]: osdmap e407: 8 total, 8 up, 8 in 2024-09-17T11:50:13.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:12 smithi129 ceph-mon[33364]: osdmap e407: 8 total, 8 up, 8 in 2024-09-17T11:50:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:13 smithi049 ceph-mon[40223]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-17T11:50:14.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:13 smithi049 ceph-mon[30017]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-17T11:50:14.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:13 smithi129 ceph-mon[33364]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-17T11:50:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:15 smithi049 ceph-mon[40223]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-17T11:50:16.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:15 smithi049 ceph-mon[30017]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-17T11:50:16.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:15 smithi129 ceph-mon[33364]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-17T11:50:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:17.990 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:17 smithi129 ceph-mon[33364]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 891 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-17T11:50:17.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:17 smithi049 ceph-mon[40223]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 891 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-17T11:50:18.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:17 smithi049 ceph-mon[30017]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 891 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-17T11:50:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:19.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:19.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:19 smithi049 ceph-mon[40223]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-17T11:50:20.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:19 smithi049 ceph-mon[30017]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-17T11:50:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:19 smithi129 ceph-mon[33364]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-17T11:50:22.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:21 smithi049 ceph-mon[40223]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-17T11:50:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:21 smithi049 ceph-mon[30017]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-17T11:50:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:21 smithi129 ceph-mon[33364]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-17T11:50:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:23 smithi049 ceph-mon[40223]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:24.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:23 smithi049 ceph-mon[30017]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:23 smithi129 ceph-mon[33364]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:25 smithi049 ceph-mon[40223]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:25 smithi049 ceph-mon[30017]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:25 smithi129 ceph-mon[33364]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[40223]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[30017]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:27 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:28.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:27 smithi129 ceph-mon[33364]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:50:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:27 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]} : dispatch 2024-09-17T11:50:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-17T11:50:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[40223]: osdmap e408: 8 total, 8 up, 8 in 2024-09-17T11:50:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-17T11:50:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[30017]: osdmap e408: 8 total, 8 up, 8 in 2024-09-17T11:50:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.1f", "id": [6, 3]}]': finished 2024-09-17T11:50:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:28 smithi129 ceph-mon[33364]: osdmap e408: 8 total, 8 up, 8 in 2024-09-17T11:50:29.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:29 smithi049 ceph-mon[40223]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-17T11:50:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:29 smithi049 ceph-mon[40223]: osdmap e409: 8 total, 8 up, 8 in 2024-09-17T11:50:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:29 smithi049 ceph-mon[30017]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-17T11:50:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:29 smithi049 ceph-mon[30017]: osdmap e409: 8 total, 8 up, 8 in 2024-09-17T11:50:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:29 smithi129 ceph-mon[33364]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-17T11:50:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:29 smithi129 ceph-mon[33364]: osdmap e409: 8 total, 8 up, 8 in 2024-09-17T11:50:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:31 smithi049 ceph-mon[40223]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:50:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:31 smithi049 ceph-mon[30017]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:50:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:31 smithi129 ceph-mon[33364]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:50:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:33.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-71"} : dispatch 2024-09-17T11:50:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:33 smithi049 ceph-mon[40223]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:50:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:33 smithi049 ceph-mon[40223]: osdmap e410: 8 total, 8 up, 8 in 2024-09-17T11:50:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:33 smithi049 ceph-mon[30017]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:50:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:33 smithi049 ceph-mon[30017]: osdmap e410: 8 total, 8 up, 8 in 2024-09-17T11:50:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:33 smithi129 ceph-mon[33364]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:50:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:33 smithi129 ceph-mon[33364]: osdmap e410: 8 total, 8 up, 8 in 2024-09-17T11:50:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[40223]: osdmap e411: 8 total, 8 up, 8 in 2024-09-17T11:50:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[30017]: osdmap e411: 8 total, 8 up, 8 in 2024-09-17T11:50:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:34 smithi129 ceph-mon[33364]: osdmap e411: 8 total, 8 up, 8 in 2024-09-17T11:50:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[30017]: pgmap v595: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[30017]: osdmap e412: 8 total, 8 up, 8 in 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[40223]: pgmap v595: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[40223]: osdmap e412: 8 total, 8 up, 8 in 2024-09-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:35 smithi129 ceph-mon[33364]: pgmap v595: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:35 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:35 smithi129 ceph-mon[33364]: osdmap e412: 8 total, 8 up, 8 in 2024-09-17T11:50:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:36.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:36 smithi049 ceph-mon[40223]: osdmap e413: 8 total, 8 up, 8 in 2024-09-17T11:50:36.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:36 smithi049 ceph-mon[30017]: osdmap e413: 8 total, 8 up, 8 in 2024-09-17T11:50:36.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:36 smithi129 ceph-mon[33364]: osdmap e413: 8 total, 8 up, 8 in 2024-09-17T11:50:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:37 smithi049 ceph-mon[30017]: pgmap v598: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:37 smithi049 ceph-mon[30017]: osdmap e414: 8 total, 8 up, 8 in 2024-09-17T11:50:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:37 smithi049 ceph-mon[40223]: pgmap v598: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:37 smithi049 ceph-mon[40223]: osdmap e414: 8 total, 8 up, 8 in 2024-09-17T11:50:38.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:37 smithi129 ceph-mon[33364]: pgmap v598: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:37 smithi129 ceph-mon[33364]: osdmap e414: 8 total, 8 up, 8 in 2024-09-17T11:50:38.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:39.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:39 smithi049 ceph-mon[40223]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:39 smithi049 ceph-mon[30017]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:39 smithi129 ceph-mon[33364]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-17T11:50:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:41 smithi049 ceph-mon[40223]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:50:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:41 smithi049 ceph-mon[30017]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:50:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:41 smithi129 ceph-mon[33364]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-17T11:50:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:43.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:43 smithi049 ceph-mon[40223]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 918 B/s wr, 4 op/s 2024-09-17T11:50:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:43 smithi049 ceph-mon[30017]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 918 B/s wr, 4 op/s 2024-09-17T11:50:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:43 smithi129 ceph-mon[33364]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 918 B/s wr, 4 op/s 2024-09-17T11:50:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:44 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:44 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:44 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[40223]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 814 B/s wr, 3 op/s 2024-09-17T11:50:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:50:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[30017]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 814 B/s wr, 3 op/s 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:50:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:45 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:50:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:45 smithi129 ceph-mon[33364]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 814 B/s wr, 3 op/s 2024-09-17T11:50:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:45 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:50:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:45 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:50:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:45 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:50:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:45 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:50:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[30017]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-17T11:50:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[40223]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:48.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-mon[33364]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-17T11:50:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-73"} : dispatch 2024-09-17T11:50:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[30017]: osdmap e415: 8 total, 8 up, 8 in 2024-09-17T11:50:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[30017]: osdmap e416: 8 total, 8 up, 8 in 2024-09-17T11:50:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:49.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[40223]: osdmap e415: 8 total, 8 up, 8 in 2024-09-17T11:50:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[40223]: osdmap e416: 8 total, 8 up, 8 in 2024-09-17T11:50:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:48 smithi129 ceph-mon[33364]: osdmap e415: 8 total, 8 up, 8 in 2024-09-17T11:50:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:48 smithi129 ceph-mon[33364]: osdmap e416: 8 total, 8 up, 8 in 2024-09-17T11:50:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[30017]: pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:50:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[30017]: osdmap e417: 8 total, 8 up, 8 in 2024-09-17T11:50:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[40223]: pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:50:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[40223]: osdmap e417: 8 total, 8 up, 8 in 2024-09-17T11:50:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:49 smithi129 ceph-mon[33364]: pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:50:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:49 smithi129 ceph-mon[33364]: osdmap e417: 8 total, 8 up, 8 in 2024-09-17T11:50:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:51 smithi049 ceph-mon[30017]: pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:51 smithi049 ceph-mon[30017]: osdmap e418: 8 total, 8 up, 8 in 2024-09-17T11:50:52.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:51 smithi049 ceph-mon[40223]: pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:52.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:51 smithi049 ceph-mon[40223]: osdmap e418: 8 total, 8 up, 8 in 2024-09-17T11:50:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:51 smithi129 ceph-mon[33364]: pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:51 smithi129 ceph-mon[33364]: osdmap e418: 8 total, 8 up, 8 in 2024-09-17T11:50:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[40223]: osdmap e419: 8 total, 8 up, 8 in 2024-09-17T11:50:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[30017]: osdmap e419: 8 total, 8 up, 8 in 2024-09-17T11:50:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:52 smithi129 ceph-mon[33364]: osdmap e419: 8 total, 8 up, 8 in 2024-09-17T11:50:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-75"} : dispatch 2024-09-17T11:50:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:53 smithi049 ceph-mon[40223]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-17T11:50:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:53 smithi049 ceph-mon[40223]: osdmap e420: 8 total, 8 up, 8 in 2024-09-17T11:50:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:53 smithi049 ceph-mon[30017]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-17T11:50:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:53 smithi049 ceph-mon[30017]: osdmap e420: 8 total, 8 up, 8 in 2024-09-17T11:50:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:53 smithi129 ceph-mon[33364]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-17T11:50:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:53 smithi129 ceph-mon[33364]: osdmap e420: 8 total, 8 up, 8 in 2024-09-17T11:50:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[40223]: osdmap e421: 8 total, 8 up, 8 in 2024-09-17T11:50:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[30017]: osdmap e421: 8 total, 8 up, 8 in 2024-09-17T11:50:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:54 smithi129 ceph-mon[33364]: osdmap e421: 8 total, 8 up, 8 in 2024-09-17T11:50:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[40223]: pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:50:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[40223]: osdmap e422: 8 total, 8 up, 8 in 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[30017]: pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[30017]: osdmap e422: 8 total, 8 up, 8 in 2024-09-17T11:50:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:55 smithi129 ceph-mon[33364]: pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-17T11:50:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:55 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:50:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:50:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:55 smithi129 ceph-mon[33364]: osdmap e422: 8 total, 8 up, 8 in 2024-09-17T11:50:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:50:56.872 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:50:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:50:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[40223]: osdmap e423: 8 total, 8 up, 8 in 2024-09-17T11:50:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[30017]: osdmap e423: 8 total, 8 up, 8 in 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:57.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:56 smithi129 ceph-mon[33364]: osdmap e423: 8 total, 8 up, 8 in 2024-09-17T11:50:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:50:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-77"} : dispatch 2024-09-17T11:50:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[40223]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[40223]: osdmap e424: 8 total, 8 up, 8 in 2024-09-17T11:50:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[30017]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[30017]: osdmap e424: 8 total, 8 up, 8 in 2024-09-17T11:50:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:58.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:50:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:50:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:57 smithi129 ceph-mon[33364]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:50:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:57 smithi129 ceph-mon[33364]: osdmap e424: 8 total, 8 up, 8 in 2024-09-17T11:50:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:50:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[40223]: osdmap e425: 8 total, 8 up, 8 in 2024-09-17T11:50:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[30017]: osdmap e425: 8 total, 8 up, 8 in 2024-09-17T11:50:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:50:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:58 smithi129 ceph-mon[33364]: osdmap e425: 8 total, 8 up, 8 in 2024-09-17T11:50:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:50:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[40223]: pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[40223]: osdmap e426: 8 total, 8 up, 8 in 2024-09-17T11:51:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[30017]: pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[30017]: osdmap e426: 8 total, 8 up, 8 in 2024-09-17T11:51:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:50:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:59 smithi129 ceph-mon[33364]: pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:59 smithi129 ceph-mon[33364]: osdmap e426: 8 total, 8 up, 8 in 2024-09-17T11:51:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:50:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:00 smithi049 ceph-mon[40223]: osdmap e427: 8 total, 8 up, 8 in 2024-09-17T11:51:01.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:00 smithi049 ceph-mon[30017]: osdmap e427: 8 total, 8 up, 8 in 2024-09-17T11:51:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:00 smithi129 ceph-mon[33364]: osdmap e427: 8 total, 8 up, 8 in 2024-09-17T11:51:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:01 smithi129 ceph-mon[33364]: pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:01 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:01 smithi129 ceph-mon[33364]: osdmap e428: 8 total, 8 up, 8 in 2024-09-17T11:51:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.7"} : dispatch 2024-09-17T11:51:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:01 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.7"}]: dispatch 2024-09-17T11:51:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[40223]: pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:02.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[40223]: osdmap e428: 8 total, 8 up, 8 in 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.7"} : dispatch 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.7"}]: dispatch 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[30017]: pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[30017]: osdmap e428: 8 total, 8 up, 8 in 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.7"} : dispatch 2024-09-17T11:51:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:01 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.7"}]: dispatch 2024-09-17T11:51:03.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:02 smithi129 ceph-mon[33364]: 304.7 deep-scrub starts 2024-09-17T11:51:03.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:02 smithi129 ceph-mon[33364]: 304.7 deep-scrub ok 2024-09-17T11:51:03.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:02 smithi049 ceph-mon[40223]: 304.7 deep-scrub starts 2024-09-17T11:51:03.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:02 smithi049 ceph-mon[40223]: 304.7 deep-scrub ok 2024-09-17T11:51:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:02 smithi049 ceph-mon[30017]: 304.7 deep-scrub starts 2024-09-17T11:51:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:02 smithi049 ceph-mon[30017]: 304.7 deep-scrub ok 2024-09-17T11:51:04.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:03 smithi129 ceph-mon[33364]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-17T11:51:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:03 smithi049 ceph-mon[40223]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-17T11:51:04.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:03 smithi049 ceph-mon[30017]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-17T11:51:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:05 smithi129 ceph-mon[33364]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-17T11:51:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:05 smithi049 ceph-mon[40223]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-17T11:51:06.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:05 smithi049 ceph-mon[30017]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-17T11:51:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:07 smithi129 ceph-mon[33364]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 663 B/s rd, 663 B/s wr, 1 op/s 2024-09-17T11:51:08.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:07 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:07 smithi049 ceph-mon[30017]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 663 B/s rd, 663 B/s wr, 1 op/s 2024-09-17T11:51:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:07 smithi049 ceph-mon[40223]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 663 B/s rd, 663 B/s wr, 1 op/s 2024-09-17T11:51:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:09.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:09.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:09 smithi129 ceph-mon[33364]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 587 B/s wr, 2 op/s 2024-09-17T11:51:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:09 smithi049 ceph-mon[40223]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 587 B/s wr, 2 op/s 2024-09-17T11:51:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:09 smithi049 ceph-mon[30017]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 587 B/s wr, 2 op/s 2024-09-17T11:51:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:11 smithi129 ceph-mon[33364]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:51:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:11 smithi049 ceph-mon[40223]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:51:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:11 smithi049 ceph-mon[30017]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:51:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:13.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:13.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:14.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:13 smithi049 ceph-mon[30017]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-09-17T11:51:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:13 smithi049 ceph-mon[40223]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-09-17T11:51:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:13 smithi129 ceph-mon[33364]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-09-17T11:51:16.202 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:15 smithi049 ceph-mon[40223]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:16.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:15 smithi049 ceph-mon[30017]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:15 smithi129 ceph-mon[33364]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:18.229 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:17 smithi129 ceph-mon[33364]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:51:18.229 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:17 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:18.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:17 smithi049 ceph-mon[30017]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:51:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:17 smithi049 ceph-mon[40223]: pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:51:19.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:19.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:19 smithi049 ceph-mon[40223]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:19 smithi049 ceph-mon[30017]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:19 smithi129 ceph-mon[33364]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:21.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:21.403 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T11:51:20.934+0000 7f949e0aa640 -1 snap_mapper.add_oid found existing snaps mapped on 304:e3f441d6:test-rados-api-smithi049-76013-80::foo:2, removing 2024-09-17T11:51:21.404 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 11:51:20 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T11:51:20.934+0000 7f34a4e98640 -1 snap_mapper.add_oid found existing snaps mapped on 304:e3f441d6:test-rados-api-smithi049-76013-80::foo:2, removing 2024-09-17T11:51:21.427 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 11:51:20 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T11:51:20.934+0000 7f539d132640 -1 snap_mapper.add_oid found existing snaps mapped on 304:e3f441d6:test-rados-api-smithi049-76013-80::foo:2, removing 2024-09-17T11:51:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:21.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:21.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-79"} : dispatch 2024-09-17T11:51:22.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:21 smithi049 ceph-mon[30017]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:22.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:21 smithi049 ceph-mon[40223]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:21 smithi129 ceph-mon[33364]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:51:23.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:22 smithi049 ceph-mon[40223]: osdmap e429: 8 total, 8 up, 8 in 2024-09-17T11:51:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:22 smithi049 ceph-mon[30017]: osdmap e429: 8 total, 8 up, 8 in 2024-09-17T11:51:23.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:22 smithi129 ceph-mon[33364]: osdmap e429: 8 total, 8 up, 8 in 2024-09-17T11:51:24.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[40223]: pgmap v637: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:24.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[40223]: osdmap e430: 8 total, 8 up, 8 in 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[30017]: pgmap v637: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[30017]: osdmap e430: 8 total, 8 up, 8 in 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:23 smithi129 ceph-mon[33364]: pgmap v637: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:23 smithi129 ceph-mon[33364]: osdmap e430: 8 total, 8 up, 8 in 2024-09-17T11:51:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:25.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[40223]: osdmap e431: 8 total, 8 up, 8 in 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[30017]: osdmap e431: 8 total, 8 up, 8 in 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:25.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:25.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:24 smithi129 ceph-mon[33364]: osdmap e431: 8 total, 8 up, 8 in 2024-09-17T11:51:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:25.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:26.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[30017]: pgmap v640: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:26.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:26.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[30017]: osdmap e432: 8 total, 8 up, 8 in 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[40223]: pgmap v640: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[40223]: osdmap e432: 8 total, 8 up, 8 in 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:25 smithi129 ceph-mon[33364]: pgmap v640: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:51:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:25 smithi129 ceph-mon[33364]: osdmap e432: 8 total, 8 up, 8 in 2024-09-17T11:51:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:27.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:27.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[30017]: osdmap e433: 8 total, 8 up, 8 in 2024-09-17T11:51:27.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[40223]: osdmap e433: 8 total, 8 up, 8 in 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:26 smithi129 ceph-mon[33364]: osdmap e433: 8 total, 8 up, 8 in 2024-09-17T11:51:27.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:27.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: pgmap v643: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: osdmap e434: 8 total, 8 up, 8 in 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[40223]: osdmap e435: 8 total, 8 up, 8 in 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: pgmap v643: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: osdmap e434: 8 total, 8 up, 8 in 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:28.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:27 smithi049 ceph-mon[30017]: osdmap e435: 8 total, 8 up, 8 in 2024-09-17T11:51:28.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: pgmap v643: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: osdmap e434: 8 total, 8 up, 8 in 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:27 smithi129 ceph-mon[33364]: osdmap e435: 8 total, 8 up, 8 in 2024-09-17T11:51:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:29.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:29.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:29.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:29.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-17T11:51:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[40223]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-17T11:51:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[40223]: osdmap e436: 8 total, 8 up, 8 in 2024-09-17T11:51:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[30017]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-17T11:51:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[30017]: osdmap e436: 8 total, 8 up, 8 in 2024-09-17T11:51:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:30.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-17T11:51:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: osdmap e436: 8 total, 8 up, 8 in 2024-09-17T11:51:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-17T11:51:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-17T11:51:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[40223]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T11:51:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-17T11:51:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[30017]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T11:51:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:31.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-17T11:51:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:31 smithi129 ceph-mon[33364]: osdmap e437: 8 total, 8 up, 8 in 2024-09-17T11:51:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:31.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:32.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: osdmap e438: 8 total, 8 up, 8 in 2024-09-17T11:51:32.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:32.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:32.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-81"} : dispatch 2024-09-17T11:51:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:33 smithi049 ceph-mon[40223]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T11:51:33.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:33 smithi049 ceph-mon[30017]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T11:51:33.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:33 smithi129 ceph-mon[33364]: osdmap e439: 8 total, 8 up, 8 in 2024-09-17T11:51:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[40223]: pgmap v652: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[40223]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T11:51:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[30017]: pgmap v652: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[30017]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T11:51:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:34 smithi129 ceph-mon[33364]: pgmap v652: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:34 smithi129 ceph-mon[33364]: osdmap e440: 8 total, 8 up, 8 in 2024-09-17T11:51:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:34.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[30017]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[40223]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:35 smithi129 ceph-mon[33364]: osdmap e441: 8 total, 8 up, 8 in 2024-09-17T11:51:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[30017]: pgmap v655: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[30017]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[40223]: pgmap v655: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[40223]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T11:51:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:36 smithi129 ceph-mon[33364]: pgmap v655: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-09-17T11:51:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:36 smithi129 ceph-mon[33364]: osdmap e442: 8 total, 8 up, 8 in 2024-09-17T11:51:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:37.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:37.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[30017]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T11:51:37.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:37.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:37.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:37.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[40223]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T11:51:37.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:37.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:37.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:37.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:37 smithi129 ceph-mon[33364]: osdmap e443: 8 total, 8 up, 8 in 2024-09-17T11:51:37.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:37.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[40223]: pgmap v658: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[40223]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T11:51:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[30017]: pgmap v658: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[30017]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:38 smithi129 ceph-mon[33364]: pgmap v658: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:38 smithi129 ceph-mon[33364]: osdmap e444: 8 total, 8 up, 8 in 2024-09-17T11:51:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:38.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[40223]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T11:51:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:39 smithi049 ceph-mon[30017]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T11:51:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:39 smithi129 ceph-mon[33364]: osdmap e445: 8 total, 8 up, 8 in 2024-09-17T11:51:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:40 smithi049 ceph-mon[40223]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:40 smithi049 ceph-mon[40223]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T11:51:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:40 smithi049 ceph-mon[30017]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:40 smithi049 ceph-mon[30017]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T11:51:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:40 smithi129 ceph-mon[33364]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:40 smithi129 ceph-mon[33364]: osdmap e446: 8 total, 8 up, 8 in 2024-09-17T11:51:41.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[40223]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T11:51:41.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[30017]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:41 smithi129 ceph-mon[33364]: osdmap e447: 8 total, 8 up, 8 in 2024-09-17T11:51:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-83"} : dispatch 2024-09-17T11:51:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:42 smithi049 ceph-mon[40223]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:42 smithi049 ceph-mon[40223]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T11:51:42.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:42 smithi049 ceph-mon[30017]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:42 smithi049 ceph-mon[30017]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T11:51:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:42 smithi129 ceph-mon[33364]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:51:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:42 smithi129 ceph-mon[33364]: osdmap e448: 8 total, 8 up, 8 in 2024-09-17T11:51:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[40223]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T11:51:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[30017]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T11:51:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:43 smithi129 ceph-mon[33364]: osdmap e449: 8 total, 8 up, 8 in 2024-09-17T11:51:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:44.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: pgmap v667: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:44.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:44.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:44.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: pgmap v667: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:44.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: pgmap v667: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: osdmap e450: 8 total, 8 up, 8 in 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:45.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:45.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:45.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[40223]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T11:51:45.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[30017]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T11:51:45.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:45.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:45 smithi129 ceph-mon[33364]: osdmap e451: 8 total, 8 up, 8 in 2024-09-17T11:51:45.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:46.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: pgmap v670: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:46.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:46.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T11:51:46.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: pgmap v670: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:51:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: pgmap v670: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-17T11:51:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_tier","val": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:51:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: osdmap e452: 8 total, 8 up, 8 in 2024-09-17T11:51:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:46 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:51:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T11:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: pgmap v673: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[30017]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:51:47.403 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:51:47.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:51:47.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: pgmap v673: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:47.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:47.404 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:47 smithi049 ceph-mon[40223]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T11:51:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: osdmap e453: 8 total, 8 up, 8 in 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: pgmap v673: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-mon[33364]: osdmap e454: 8 total, 8 up, 8 in 2024-09-17T11:51:48.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:48 smithi129 ceph-mon[33364]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T11:51:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:48 smithi049 ceph-mon[40223]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T11:51:48.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:48.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:48 smithi049 ceph-mon[30017]: osdmap e455: 8 total, 8 up, 8 in 2024-09-17T11:51:49.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:49 smithi049 ceph-mon[30017]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:51:49.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:49 smithi049 ceph-mon[30017]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T11:51:49.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:49 smithi049 ceph-mon[40223]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:51:49.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:49 smithi049 ceph-mon[40223]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T11:51:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:49 smithi129 ceph-mon[33364]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:51:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:49 smithi129 ceph-mon[33364]: osdmap e456: 8 total, 8 up, 8 in 2024-09-17T11:51:50.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:50 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:50.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:50.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-85"} : dispatch 2024-09-17T11:51:50.430 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:50 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:51.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:51 smithi129 ceph-mon[33364]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T11:51:51.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:51 smithi129 ceph-mon[33364]: pgmap v679: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:51:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:51 smithi049 ceph-mon[40223]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T11:51:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:51 smithi049 ceph-mon[40223]: pgmap v679: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:51:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:51 smithi049 ceph-mon[30017]: osdmap e457: 8 total, 8 up, 8 in 2024-09-17T11:51:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:51 smithi049 ceph-mon[30017]: pgmap v679: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:51:52.187 INFO:tasks.workunit.client.0.smithi049.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18079 ms) 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22796 ms) 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3028 ms) 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3024 ms) 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3025 ms) 2024-09-17T11:51:52.188 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3015 ms) 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4039 ms) 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39092 ms) 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16982 ms) 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-17T11:51:52.189 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4025 ms) 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3012 ms) 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25095 ms) 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15047 ms) 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.190 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5020 ms) 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4028 ms) 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: waiting for scrubs... 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: done waiting 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25079 ms) 2024-09-17T11:51:52.191 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10067 ms) 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9069 ms) 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9062 ms) 2024-09-17T11:51:52.192 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-17T11:51:52.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:52 smithi129 ceph-mon[33364]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T11:51:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[30017]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T11:51:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[40223]: osdmap e458: 8 total, 8 up, 8 in 2024-09-17T11:51:52.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:51:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[30017]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T11:51:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[30017]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-17T11:51:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[40223]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T11:51:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:53 smithi049 ceph-mon[40223]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-17T11:51:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:51:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:53 smithi129 ceph-mon[33364]: osdmap e459: 8 total, 8 up, 8 in 2024-09-17T11:51:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:51:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:53 smithi129 ceph-mon[33364]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-17T11:51:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T11:51:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T11:51:54.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:54.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: osdmap e460: 8 total, 8 up, 8 in 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: osdmap e461: 8 total, 8 up, 8 in 2024-09-17T11:51:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"} : dispatch 2024-09-17T11:51:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[40223]: pgmap v685: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-17T11:51:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"}]': finished 2024-09-17T11:51:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[40223]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T11:51:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[30017]: pgmap v685: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"}]': finished 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[30017]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:55 smithi129 ceph-mon[33364]: pgmap v685: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-17T11:51:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_tier","val": "test-rados-api-smithi049-76013-89-test-flush"}]': finished 2024-09-17T11:51:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:55 smithi129 ceph-mon[33364]: osdmap e462: 8 total, 8 up, 8 in 2024-09-17T11:51:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:51:56.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[40223]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[30017]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T11:51:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:56 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:51:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:51:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:56 smithi129 ceph-mon[33364]: osdmap e463: 8 total, 8 up, 8 in 2024-09-17T11:51:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:51:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:51:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:51:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:51:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[40223]: pgmap v688: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[40223]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T11:51:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[30017]: pgmap v688: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:57 smithi049 ceph-mon[30017]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T11:51:57.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:57 smithi129 ceph-mon[33364]: pgmap v688: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:51:57.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:51:57.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:57 smithi129 ceph-mon[33364]: osdmap e464: 8 total, 8 up, 8 in 2024-09-17T11:51:58.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:58 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:58.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:51:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:51:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:58 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:58 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:51:58.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:51:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T11:51:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[40223]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:51:59 smithi049 ceph-mon[30017]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:51:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: osdmap e465: 8 total, 8 up, 8 in 2024-09-17T11:51:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:51:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-87"} : dispatch 2024-09-17T11:51:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:51:59 smithi129 ceph-mon[33364]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-17T11:52:00.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:00 smithi049 ceph-mon[40223]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T11:52:00.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:00 smithi049 ceph-mon[30017]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T11:52:00.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:00 smithi129 ceph-mon[33364]: osdmap e466: 8 total, 8 up, 8 in 2024-09-17T11:52:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[40223]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T11:52:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[40223]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:52:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[30017]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T11:52:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:01 smithi049 ceph-mon[30017]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:52:01.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:01 smithi129 ceph-mon[33364]: osdmap e467: 8 total, 8 up, 8 in 2024-09-17T11:52:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:01 smithi129 ceph-mon[33364]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:52:02.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:02.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T11:52:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:02.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:02.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: osdmap e468: 8 total, 8 up, 8 in 2024-09-17T11:52:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:02.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:52:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:52:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[40223]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T11:52:03.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[40223]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-17T11:52:03.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:52:03.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[30017]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T11:52:03.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:03 smithi049 ceph-mon[30017]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-17T11:52:03.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:52:03.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:03 smithi129 ceph-mon[33364]: osdmap e469: 8 total, 8 up, 8 in 2024-09-17T11:52:03.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:03 smithi129 ceph-mon[33364]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-17T11:52:04.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:04 smithi049 ceph-mon[40223]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T11:52:04.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:04 smithi049 ceph-mon[30017]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T11:52:04.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:04 smithi129 ceph-mon[33364]: osdmap e470: 8 total, 8 up, 8 in 2024-09-17T11:52:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: osdmap e471: 8 total, 8 up, 8 in 2024-09-17T11:52:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[40223]: pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: osdmap e471: 8 total, 8 up, 8 in 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:05 smithi049 ceph-mon[30017]: pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:52:05.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: osdmap e471: 8 total, 8 up, 8 in 2024-09-17T11:52:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-90"} : dispatch 2024-09-17T11:52:05.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:05 smithi129 ceph-mon[33364]: pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-17T11:52:06.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:06 smithi049 ceph-mon[40223]: osdmap e472: 8 total, 8 up, 8 in 2024-09-17T11:52:06.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:06 smithi049 ceph-mon[30017]: osdmap e472: 8 total, 8 up, 8 in 2024-09-17T11:52:06.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:06 smithi129 ceph-mon[33364]: osdmap e472: 8 total, 8 up, 8 in 2024-09-17T11:52:06.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[30017]: osdmap e473: 8 total, 8 up, 8 in 2024-09-17T11:52:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[30017]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:07.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[40223]: osdmap e473: 8 total, 8 up, 8 in 2024-09-17T11:52:07.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:07 smithi049 ceph-mon[40223]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:07.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:07 smithi129 ceph-mon[33364]: osdmap e473: 8 total, 8 up, 8 in 2024-09-17T11:52:07.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:07.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:07 smithi129 ceph-mon[33364]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:08.273 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[40223]: osdmap e474: 8 total, 8 up, 8 in 2024-09-17T11:52:08.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[30017]: osdmap e474: 8 total, 8 up, 8 in 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:08.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:08.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-mon[33364]: osdmap e474: 8 total, 8 up, 8 in 2024-09-17T11:52:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:09 smithi049 ceph-mon[30017]: osdmap e475: 8 total, 8 up, 8 in 2024-09-17T11:52:09.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:09 smithi049 ceph-mon[30017]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 179 op/s 2024-09-17T11:52:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:09 smithi049 ceph-mon[40223]: osdmap e475: 8 total, 8 up, 8 in 2024-09-17T11:52:09.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:09 smithi049 ceph-mon[40223]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 179 op/s 2024-09-17T11:52:09.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:09 smithi129 ceph-mon[33364]: osdmap e475: 8 total, 8 up, 8 in 2024-09-17T11:52:09.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:09 smithi129 ceph-mon[33364]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 179 op/s 2024-09-17T11:52:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[30017]: osdmap e476: 8 total, 8 up, 8 in 2024-09-17T11:52:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:10.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[40223]: osdmap e476: 8 total, 8 up, 8 in 2024-09-17T11:52:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:10.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:10.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:10 smithi129 ceph-mon[33364]: osdmap e476: 8 total, 8 up, 8 in 2024-09-17T11:52:10.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:10.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:10.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-92"} : dispatch 2024-09-17T11:52:11.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:11 smithi049 ceph-mon[40223]: osdmap e477: 8 total, 8 up, 8 in 2024-09-17T11:52:11.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:11 smithi049 ceph-mon[40223]: pgmap v709: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.5 KiB/s wr, 181 op/s 2024-09-17T11:52:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:11 smithi049 ceph-mon[30017]: osdmap e477: 8 total, 8 up, 8 in 2024-09-17T11:52:11.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:11 smithi049 ceph-mon[30017]: pgmap v709: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.5 KiB/s wr, 181 op/s 2024-09-17T11:52:11.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:11 smithi129 ceph-mon[33364]: osdmap e477: 8 total, 8 up, 8 in 2024-09-17T11:52:11.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:11 smithi129 ceph-mon[33364]: pgmap v709: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.5 KiB/s wr, 181 op/s 2024-09-17T11:52:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[40223]: osdmap e478: 8 total, 8 up, 8 in 2024-09-17T11:52:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:12.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[30017]: osdmap e478: 8 total, 8 up, 8 in 2024-09-17T11:52:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:12.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:12.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:12 smithi129 ceph-mon[33364]: osdmap e478: 8 total, 8 up, 8 in 2024-09-17T11:52:12.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:12.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:13.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[40223]: osdmap e479: 8 total, 8 up, 8 in 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[40223]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[30017]: osdmap e479: 8 total, 8 up, 8 in 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:13.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:13 smithi049 ceph-mon[30017]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:52:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:13 smithi129 ceph-mon[33364]: osdmap e479: 8 total, 8 up, 8 in 2024-09-17T11:52:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:13.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:13 smithi129 ceph-mon[33364]: pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:52:14.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:14 smithi049 ceph-mon[40223]: osdmap e480: 8 total, 8 up, 8 in 2024-09-17T11:52:14.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:14 smithi049 ceph-mon[30017]: osdmap e480: 8 total, 8 up, 8 in 2024-09-17T11:52:14.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:14 smithi129 ceph-mon[33364]: osdmap e480: 8 total, 8 up, 8 in 2024-09-17T11:52:15.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[30017]: osdmap e481: 8 total, 8 up, 8 in 2024-09-17T11:52:15.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:15.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[30017]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:52:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[40223]: osdmap e481: 8 total, 8 up, 8 in 2024-09-17T11:52:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:15.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:15 smithi049 ceph-mon[40223]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:52:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:15 smithi129 ceph-mon[33364]: osdmap e481: 8 total, 8 up, 8 in 2024-09-17T11:52:15.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:15 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:52:15.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:15 smithi129 ceph-mon[33364]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-17T11:52:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:18.012 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:17 smithi129 ceph-mon[33364]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:52:18.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:17 smithi049 ceph-mon[30017]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:52:18.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:17 smithi049 ceph-mon[40223]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-17T11:52:18.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:19.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:19.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:20.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:19 smithi049 ceph-mon[30017]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-17T11:52:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:19 smithi049 ceph-mon[40223]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-17T11:52:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:19 smithi129 ceph-mon[33364]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-17T11:52:22.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:21 smithi049 ceph-mon[40223]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-17T11:52:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:21 smithi049 ceph-mon[30017]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-17T11:52:22.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:21 smithi129 ceph-mon[33364]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-17T11:52:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:23 smithi049 ceph-mon[40223]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 219 B/s wr, 3 op/s 2024-09-17T11:52:24.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:23 smithi049 ceph-mon[30017]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 219 B/s wr, 3 op/s 2024-09-17T11:52:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:23 smithi129 ceph-mon[33364]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 219 B/s wr, 3 op/s 2024-09-17T11:52:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:25 smithi049 ceph-mon[30017]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 198 B/s wr, 2 op/s 2024-09-17T11:52:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:25 smithi049 ceph-mon[30017]: osdmap e482: 8 total, 8 up, 8 in 2024-09-17T11:52:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:25 smithi049 ceph-mon[40223]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 198 B/s wr, 2 op/s 2024-09-17T11:52:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:25 smithi049 ceph-mon[40223]: osdmap e482: 8 total, 8 up, 8 in 2024-09-17T11:52:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:25 smithi129 ceph-mon[33364]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 198 B/s wr, 2 op/s 2024-09-17T11:52:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:25 smithi129 ceph-mon[33364]: osdmap e482: 8 total, 8 up, 8 in 2024-09-17T11:52:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:28.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:27 smithi129 ceph-mon[33364]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-17T11:52:28.019 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:27 smithi049 ceph-mon[30017]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-17T11:52:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:27 smithi049 ceph-mon[40223]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-17T11:52:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:28.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:30.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:29 smithi049 ceph-mon[30017]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:30.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:29 smithi049 ceph-mon[30017]: osdmap e483: 8 total, 8 up, 8 in 2024-09-17T11:52:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:29 smithi049 ceph-mon[40223]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:29 smithi049 ceph-mon[40223]: osdmap e483: 8 total, 8 up, 8 in 2024-09-17T11:52:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:29 smithi129 ceph-mon[33364]: pgmap v723: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:29 smithi129 ceph-mon[33364]: osdmap e483: 8 total, 8 up, 8 in 2024-09-17T11:52:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:31 smithi049 ceph-mon[40223]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:31 smithi049 ceph-mon[30017]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:31 smithi129 ceph-mon[33364]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:33 smithi049 ceph-mon[30017]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:33 smithi049 ceph-mon[40223]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:33 smithi129 ceph-mon[33364]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[40223]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[30017]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:35 smithi129 ceph-mon[33364]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:52:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-94"} : dispatch 2024-09-17T11:52:36.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:36 smithi049 ceph-mon[30017]: osdmap e484: 8 total, 8 up, 8 in 2024-09-17T11:52:36.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:36 smithi049 ceph-mon[40223]: osdmap e484: 8 total, 8 up, 8 in 2024-09-17T11:52:36.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:36 smithi129 ceph-mon[33364]: osdmap e484: 8 total, 8 up, 8 in 2024-09-17T11:52:38.029 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:37 smithi129 ceph-mon[33364]: pgmap v729: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:38.029 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:37 smithi129 ceph-mon[33364]: osdmap e485: 8 total, 8 up, 8 in 2024-09-17T11:52:38.030 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.030 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[40223]: pgmap v729: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[40223]: osdmap e485: 8 total, 8 up, 8 in 2024-09-17T11:52:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[30017]: pgmap v729: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[30017]: osdmap e485: 8 total, 8 up, 8 in 2024-09-17T11:52:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:38.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[40223]: osdmap e486: 8 total, 8 up, 8 in 2024-09-17T11:52:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[30017]: osdmap e486: 8 total, 8 up, 8 in 2024-09-17T11:52:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:38 smithi129 ceph-mon[33364]: osdmap e486: 8 total, 8 up, 8 in 2024-09-17T11:52:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:52:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:39 smithi049 ceph-mon[30017]: pgmap v732: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:39 smithi049 ceph-mon[30017]: osdmap e487: 8 total, 8 up, 8 in 2024-09-17T11:52:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:39 smithi049 ceph-mon[40223]: pgmap v732: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:39 smithi049 ceph-mon[40223]: osdmap e487: 8 total, 8 up, 8 in 2024-09-17T11:52:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:39 smithi129 ceph-mon[33364]: pgmap v732: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:39 smithi129 ceph-mon[33364]: osdmap e487: 8 total, 8 up, 8 in 2024-09-17T11:52:42.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:41 smithi049 ceph-mon[30017]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-17T11:52:42.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:41 smithi049 ceph-mon[40223]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-17T11:52:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:41 smithi129 ceph-mon[33364]: pgmap v734: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 208 B/s wr, 1 op/s 2024-09-17T11:52:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:43.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:43 smithi049 ceph-mon[40223]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:52:44.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:43 smithi049 ceph-mon[30017]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:52:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:43 smithi129 ceph-mon[33364]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-17T11:52:45.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:45 smithi049 ceph-mon[40223]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 777 B/s wr, 2 op/s 2024-09-17T11:52:45.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:45 smithi049 ceph-mon[30017]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 777 B/s wr, 2 op/s 2024-09-17T11:52:45.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:45 smithi129 ceph-mon[33364]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 777 B/s wr, 2 op/s 2024-09-17T11:52:46.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:46 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:52:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:46 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:52:46.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:46 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:52:47.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 575 B/s wr, 1 op/s 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi129 to 3248M 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: Adjusting osd_memory_target on smithi049 to 3248M 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:52:47.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 575 B/s wr, 1 op/s 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi129 to 3248M 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: Adjusting osd_memory_target on smithi049 to 3248M 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:52:47.904 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 575 B/s wr, 1 op/s 2024-09-17T11:52:47.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi129 to 3248M 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: Adjusting osd_memory_target on smithi049 to 3248M 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:52:47.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:52:48.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:48.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:48.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:48.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:48.901 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 11:52:48 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T11:52:48.778+0000 7ff1f78d2640 -1 snap_mapper.add_oid found existing snaps mapped on 107:24337c46:test-rados-api-smithi049-76013-97::foo:21, removing 2024-09-17T11:52:49.177 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 11:52:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T11:52:48.779+0000 7f3cf1b94640 -1 snap_mapper.add_oid found existing snaps mapped on 107:24337c46:test-rados-api-smithi049-76013-97::foo:21, removing 2024-09-17T11:52:49.177 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 11:52:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T11:52:48.779+0000 7f9da9161640 -1 snap_mapper.add_oid found existing snaps mapped on 107:24337c46:test-rados-api-smithi049-76013-97::foo:21, removing 2024-09-17T11:52:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[40223]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[30017]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:49.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:49 smithi129 ceph-mon[33364]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:49.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:52:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:49.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-96"} : dispatch 2024-09-17T11:52:50.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:50 smithi049 ceph-mon[40223]: osdmap e488: 8 total, 8 up, 8 in 2024-09-17T11:52:50.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:50 smithi049 ceph-mon[30017]: osdmap e488: 8 total, 8 up, 8 in 2024-09-17T11:52:50.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:50 smithi129 ceph-mon[33364]: osdmap e488: 8 total, 8 up, 8 in 2024-09-17T11:52:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:51 smithi129 ceph-mon[33364]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-17T11:52:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:51 smithi129 ceph-mon[33364]: osdmap e489: 8 total, 8 up, 8 in 2024-09-17T11:52:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[30017]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[30017]: osdmap e489: 8 total, 8 up, 8 in 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[40223]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[40223]: osdmap e489: 8 total, 8 up, 8 in 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:52:52.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:52.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:52 smithi129 ceph-mon[33364]: osdmap e490: 8 total, 8 up, 8 in 2024-09-17T11:52:52.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:52.929 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[40223]: osdmap e490: 8 total, 8 up, 8 in 2024-09-17T11:52:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:52:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[30017]: osdmap e490: 8 total, 8 up, 8 in 2024-09-17T11:52:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:52:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:53 smithi129 ceph-mon[33364]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-17T11:52:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:52:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:53 smithi129 ceph-mon[33364]: osdmap e491: 8 total, 8 up, 8 in 2024-09-17T11:52:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[40223]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-17T11:52:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[40223]: osdmap e491: 8 total, 8 up, 8 in 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[30017]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[30017]: osdmap e491: 8 total, 8 up, 8 in 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"} : dispatch 2024-09-17T11:52:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:54 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:52:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"}]': finished 2024-09-17T11:52:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:54 smithi129 ceph-mon[33364]: osdmap e492: 8 total, 8 up, 8 in 2024-09-17T11:52:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:52:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"}]': finished 2024-09-17T11:52:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[40223]: osdmap e492: 8 total, 8 up, 8 in 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-98", "mode": "writeback"}]': finished 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[30017]: osdmap e492: 8 total, 8 up, 8 in 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:52:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[40223]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:52:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[40223]: osdmap e493: 8 total, 8 up, 8 in 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[30017]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[30017]: osdmap e493: 8 total, 8 up, 8 in 2024-09-17T11:52:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:55 smithi129 ceph-mon[33364]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:52:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:52:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:55 smithi129 ceph-mon[33364]: osdmap e493: 8 total, 8 up, 8 in 2024-09-17T11:52:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:52:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:52:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[30017]: osdmap e494: 8 total, 8 up, 8 in 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[40223]: osdmap e494: 8 total, 8 up, 8 in 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:56 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:52:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:52:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:56 smithi129 ceph-mon[33364]: osdmap e494: 8 total, 8 up, 8 in 2024-09-17T11:52:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"}]': finished 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: osdmap e495: 8 total, 8 up, 8 in 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.035 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"}]': finished 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: osdmap e495: 8 total, 8 up, 8 in 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_count","val": "1"}]': finished 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: osdmap e495: 8 total, 8 up, 8 in 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:52:58.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:52:58.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:52:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:52:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: osdmap e496: 8 total, 8 up, 8 in 2024-09-17T11:52:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"}]': finished 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: osdmap e496: 8 total, 8 up, 8 in 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"}]': finished 2024-09-17T11:52:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:52:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: osdmap e496: 8 total, 8 up, 8 in 2024-09-17T11:52:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-17T11:52:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:52:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-98","var": "target_max_objects","val": "250"}]': finished 2024-09-17T11:53:00.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[30017]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:00.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[30017]: osdmap e497: 8 total, 8 up, 8 in 2024-09-17T11:53:00.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:00.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[40223]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[40223]: osdmap e497: 8 total, 8 up, 8 in 2024-09-17T11:53:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:52:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:59 smithi129 ceph-mon[33364]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:59 smithi129 ceph-mon[33364]: osdmap e497: 8 total, 8 up, 8 in 2024-09-17T11:53:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:52:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[40223]: osdmap e498: 8 total, 8 up, 8 in 2024-09-17T11:53:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[30017]: osdmap e498: 8 total, 8 up, 8 in 2024-09-17T11:53:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:00 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:00 smithi129 ceph-mon[33364]: osdmap e498: 8 total, 8 up, 8 in 2024-09-17T11:53:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"} : dispatch 2024-09-17T11:53:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[40223]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:53:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[40223]: osdmap e499: 8 total, 8 up, 8 in 2024-09-17T11:53:02.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[30017]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:02.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:53:02.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:01 smithi049 ceph-mon[30017]: osdmap e499: 8 total, 8 up, 8 in 2024-09-17T11:53:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:01 smithi129 ceph-mon[33364]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-98"}]': finished 2024-09-17T11:53:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:01 smithi129 ceph-mon[33364]: osdmap e499: 8 total, 8 up, 8 in 2024-09-17T11:53:03.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:02 smithi049 ceph-mon[40223]: osdmap e500: 8 total, 8 up, 8 in 2024-09-17T11:53:03.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:02 smithi049 ceph-mon[30017]: osdmap e500: 8 total, 8 up, 8 in 2024-09-17T11:53:03.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:02 smithi129 ceph-mon[33364]: osdmap e500: 8 total, 8 up, 8 in 2024-09-17T11:53:03.733 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=cleanup t=2024-09-17T11:53:03.451288801Z level=info msg="Completed cleanup jobs" duration=2.950088ms 2024-09-17T11:53:03.734 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=plugins.update.checker t=2024-09-17T11:53:03.602646639Z level=info msg="Update check succeeded" duration=63.872212ms 2024-09-17T11:53:03.734 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafana.update.checker t=2024-09-17T11:53:03.607159494Z level=info msg="Update check succeeded" duration=68.524312ms 2024-09-17T11:53:04.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[40223]: pgmap v758: 196 pgs: 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[40223]: osdmap e501: 8 total, 8 up, 8 in 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[30017]: pgmap v758: 196 pgs: 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[30017]: osdmap e501: 8 total, 8 up, 8 in 2024-09-17T11:53:04.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:04.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:04.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-mon[33364]: pgmap v758: 196 pgs: 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:04.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-mon[33364]: osdmap e501: 8 total, 8 up, 8 in 2024-09-17T11:53:04.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:04.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[30017]: osdmap e502: 8 total, 8 up, 8 in 2024-09-17T11:53:05.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:05.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[40223]: osdmap e502: 8 total, 8 up, 8 in 2024-09-17T11:53:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:05.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:05.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:05.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:04 smithi129 ceph-mon[33364]: osdmap e502: 8 total, 8 up, 8 in 2024-09-17T11:53:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:53:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[40223]: pgmap v761: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:53:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[40223]: osdmap e503: 8 total, 8 up, 8 in 2024-09-17T11:53:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[30017]: pgmap v761: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[30017]: osdmap e503: 8 total, 8 up, 8 in 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:05 smithi129 ceph-mon[33364]: pgmap v761: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:53:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:05 smithi129 ceph-mon[33364]: osdmap e503: 8 total, 8 up, 8 in 2024-09-17T11:53:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: osdmap e504: 8 total, 8 up, 8 in 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"}]': finished 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[40223]: osdmap e505: 8 total, 8 up, 8 in 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: osdmap e504: 8 total, 8 up, 8 in 2024-09-17T11:53:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"}]': finished 2024-09-17T11:53:06.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:06 smithi049 ceph-mon[30017]: osdmap e505: 8 total, 8 up, 8 in 2024-09-17T11:53:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-6", "overlaypool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: osdmap e504: 8 total, 8 up, 8 in 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"} : dispatch 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-100", "mode": "writeback"}]': finished 2024-09-17T11:53:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:06 smithi129 ceph-mon[33364]: osdmap e505: 8 total, 8 up, 8 in 2024-09-17T11:53:08.046 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:07 smithi129 ceph-mon[33364]: pgmap v764: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:08.046 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.046 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[40223]: pgmap v764: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[30017]: pgmap v764: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:08.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[40223]: osdmap e506: 8 total, 8 up, 8 in 2024-09-17T11:53:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[30017]: osdmap e506: 8 total, 8 up, 8 in 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:08 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:09.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-mon[33364]: osdmap e506: 8 total, 8 up, 8 in 2024-09-17T11:53:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:08 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:10.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[30017]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:10.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[30017]: osdmap e507: 8 total, 8 up, 8 in 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[40223]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[40223]: osdmap e507: 8 total, 8 up, 8 in 2024-09-17T11:53:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:09 smithi129 ceph-mon[33364]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:09 smithi129 ceph-mon[33364]: osdmap e507: 8 total, 8 up, 8 in 2024-09-17T11:53:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: osdmap e508: 8 total, 8 up, 8 in 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-17T11:53:11.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[40223]: osdmap e509: 8 total, 8 up, 8 in 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: osdmap e508: 8 total, 8 up, 8 in 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-17T11:53:11.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:10 smithi049 ceph-mon[30017]: osdmap e509: 8 total, 8 up, 8 in 2024-09-17T11:53:11.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:11.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:11.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: osdmap e508: 8 total, 8 up, 8 in 2024-09-17T11:53:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-17T11:53:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-17T11:53:11.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:10 smithi129 ceph-mon[33364]: osdmap e509: 8 total, 8 up, 8 in 2024-09-17T11:53:12.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[40223]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:12.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:12.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[30017]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:12.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:12.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:11 smithi129 ceph-mon[33364]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:12.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:12.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[30017]: osdmap e510: 8 total, 8 up, 8 in 2024-09-17T11:53:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:13.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:13.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[40223]: osdmap e510: 8 total, 8 up, 8 in 2024-09-17T11:53:13.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"}]': finished 2024-09-17T11:53:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:12 smithi129 ceph-mon[33364]: osdmap e510: 8 total, 8 up, 8 in 2024-09-17T11:53:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"} : dispatch 2024-09-17T11:53:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[40223]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:53:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[40223]: osdmap e511: 8 total, 8 up, 8 in 2024-09-17T11:53:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[30017]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:53:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:13 smithi049 ceph-mon[30017]: osdmap e511: 8 total, 8 up, 8 in 2024-09-17T11:53:14.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:13 smithi129 ceph-mon[33364]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:53:14.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-100"}]': finished 2024-09-17T11:53:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:13 smithi129 ceph-mon[33364]: osdmap e511: 8 total, 8 up, 8 in 2024-09-17T11:53:15.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[40223]: osdmap e512: 8 total, 8 up, 8 in 2024-09-17T11:53:15.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[40223]: osdmap e513: 8 total, 8 up, 8 in 2024-09-17T11:53:15.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:15.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[30017]: osdmap e512: 8 total, 8 up, 8 in 2024-09-17T11:53:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[30017]: osdmap e513: 8 total, 8 up, 8 in 2024-09-17T11:53:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:15.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:15.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:14 smithi129 ceph-mon[33364]: osdmap e512: 8 total, 8 up, 8 in 2024-09-17T11:53:15.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:14 smithi129 ceph-mon[33364]: osdmap e513: 8 total, 8 up, 8 in 2024-09-17T11:53:15.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:15.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[40223]: pgmap v776: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[40223]: osdmap e514: 8 total, 8 up, 8 in 2024-09-17T11:53:16.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:53:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[30017]: pgmap v776: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[30017]: osdmap e514: 8 total, 8 up, 8 in 2024-09-17T11:53:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:53:16.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:15 smithi129 ceph-mon[33364]: pgmap v776: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:16.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:16.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:15 smithi129 ceph-mon[33364]: osdmap e514: 8 total, 8 up, 8 in 2024-09-17T11:53:16.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-17T11:53:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:16.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: osdmap e515: 8 total, 8 up, 8 in 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: osdmap e515: 8 total, 8 up, 8 in 2024-09-17T11:53:16.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:16.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:17.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-17T11:53:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-17T11:53:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: osdmap e515: 8 total, 8 up, 8 in 2024-09-17T11:53:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:17.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[40223]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[40223]: osdmap e516: 8 total, 8 up, 8 in 2024-09-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[30017]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:53:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[30017]: osdmap e516: 8 total, 8 up, 8 in 2024-09-17T11:53:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:18.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:18.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:17 smithi129 ceph-mon[33364]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:18.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-17T11:53:18.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:17 smithi129 ceph-mon[33364]: osdmap e516: 8 total, 8 up, 8 in 2024-09-17T11:53:18.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:18.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-17T11:53:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:53:19.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[40223]: osdmap e517: 8 total, 8 up, 8 in 2024-09-17T11:53:19.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:19.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:53:19.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:18 smithi049 ceph-mon[30017]: osdmap e517: 8 total, 8 up, 8 in 2024-09-17T11:53:19.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:19.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-17T11:53:19.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:18 smithi129 ceph-mon[33364]: osdmap e517: 8 total, 8 up, 8 in 2024-09-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[40223]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[30017]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:19 smithi129 ceph-mon[33364]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-6"} : dispatch 2024-09-17T11:53:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1231952944' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-6", "tierpool": "test-rados-api-smithi049-76013-102"} : dispatch 2024-09-17T11:53:21.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:20 smithi049 ceph-mon[30017]: osdmap e518: 8 total, 8 up, 8 in 2024-09-17T11:53:21.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:20 smithi049 ceph-mon[40223]: osdmap e518: 8 total, 8 up, 8 in 2024-09-17T11:53:21.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:20 smithi129 ceph-mon[33364]: osdmap e518: 8 total, 8 up, 8 in 2024-09-17T11:53:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: pgmap v785: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: osdmap e519: 8 total, 8 up, 8 in 2024-09-17T11:53:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: pgmap v785: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: osdmap e519: 8 total, 8 up, 8 in 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:22.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:22.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: pgmap v785: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: osdmap e519: 8 total, 8 up, 8 in 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:22.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:23.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:23.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[40223]: osdmap e520: 8 total, 8 up, 8 in 2024-09-17T11:53:23.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:23.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:23.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:23.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[30017]: osdmap e520: 8 total, 8 up, 8 in 2024-09-17T11:53:23.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:23.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:23.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:23.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:22 smithi129 ceph-mon[33364]: osdmap e520: 8 total, 8 up, 8 in 2024-09-17T11:53:23.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:23.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:24.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:23 smithi049 ceph-mon[30017]: pgmap v788: 164 pgs: 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:24.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:23 smithi049 ceph-mon[30017]: osdmap e521: 8 total, 8 up, 8 in 2024-09-17T11:53:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:23 smithi049 ceph-mon[40223]: pgmap v788: 164 pgs: 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:24.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:23 smithi049 ceph-mon[40223]: osdmap e521: 8 total, 8 up, 8 in 2024-09-17T11:53:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:23 smithi129 ceph-mon[33364]: pgmap v788: 164 pgs: 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:23 smithi129 ceph-mon[33364]: osdmap e521: 8 total, 8 up, 8 in 2024-09-17T11:53:25.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:25.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[40223]: osdmap e522: 8 total, 8 up, 8 in 2024-09-17T11:53:25.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[40223]: osdmap e523: 8 total, 8 up, 8 in 2024-09-17T11:53:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[30017]: osdmap e522: 8 total, 8 up, 8 in 2024-09-17T11:53:25.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:24 smithi049 ceph-mon[30017]: osdmap e523: 8 total, 8 up, 8 in 2024-09-17T11:53:25.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi049-76013-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:24 smithi129 ceph-mon[33364]: osdmap e522: 8 total, 8 up, 8 in 2024-09-17T11:53:25.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:24 smithi129 ceph-mon[33364]: osdmap e523: 8 total, 8 up, 8 in 2024-09-17T11:53:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[40223]: pgmap v791: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[40223]: osdmap e524: 8 total, 8 up, 8 in 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[30017]: pgmap v791: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[30017]: osdmap e524: 8 total, 8 up, 8 in 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:25 smithi129 ceph-mon[33364]: pgmap v791: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:25 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:25 smithi129 ceph-mon[33364]: osdmap e524: 8 total, 8 up, 8 in 2024-09-17T11:53:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[30017]: pgmap v794: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[30017]: osdmap e525: 8 total, 8 up, 8 in 2024-09-17T11:53:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[40223]: pgmap v794: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[40223]: osdmap e525: 8 total, 8 up, 8 in 2024-09-17T11:53:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:28.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:27 smithi129 ceph-mon[33364]: pgmap v794: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:27 smithi129 ceph-mon[33364]: osdmap e525: 8 total, 8 up, 8 in 2024-09-17T11:53:28.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[30017]: osdmap e526: 8 total, 8 up, 8 in 2024-09-17T11:53:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[40223]: osdmap e526: 8 total, 8 up, 8 in 2024-09-17T11:53:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:29.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:28 smithi129 ceph-mon[33364]: osdmap e526: 8 total, 8 up, 8 in 2024-09-17T11:53:29.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:29.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[40223]: pgmap v797: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[40223]: osdmap e527: 8 total, 8 up, 8 in 2024-09-17T11:53:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[30017]: pgmap v797: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[30017]: osdmap e527: 8 total, 8 up, 8 in 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:29 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:29 smithi129 ceph-mon[33364]: pgmap v797: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:29 smithi129 ceph-mon[33364]: osdmap e527: 8 total, 8 up, 8 in 2024-09-17T11:53:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:29 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:31.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:30 smithi129 ceph-mon[33364]: osdmap e528: 8 total, 8 up, 8 in 2024-09-17T11:53:31.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:30 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[40223]: osdmap e528: 8 total, 8 up, 8 in 2024-09-17T11:53:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[30017]: osdmap e528: 8 total, 8 up, 8 in 2024-09-17T11:53:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:31.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:30 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: pgmap v800: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: osdmap e529: 8 total, 8 up, 8 in 2024-09-17T11:53:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:31 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:32.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: pgmap v800: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:32.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:32.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: osdmap e529: 8 total, 8 up, 8 in 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: pgmap v800: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-107", "overlaypool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: osdmap e529: 8 total, 8 up, 8 in 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:31 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:33.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:32 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:32 smithi129 ceph-mon[33364]: osdmap e530: 8 total, 8 up, 8 in 2024-09-17T11:53:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:32 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:33.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:32 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:33.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:33.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:33.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[40223]: osdmap e530: 8 total, 8 up, 8 in 2024-09-17T11:53:33.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:33.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:33.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:33.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-107-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:33.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[30017]: osdmap e530: 8 total, 8 up, 8 in 2024-09-17T11:53:33.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:33.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:32 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"} : dispatch 2024-09-17T11:53:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:33 smithi129 ceph-mon[33364]: pgmap v803: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"}]': finished 2024-09-17T11:53:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:33 smithi129 ceph-mon[33364]: osdmap e531: 8 total, 8 up, 8 in 2024-09-17T11:53:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:34.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[40223]: pgmap v803: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"}]': finished 2024-09-17T11:53:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[40223]: osdmap e531: 8 total, 8 up, 8 in 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[30017]: pgmap v803: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-107"}]': finished 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[30017]: osdmap e531: 8 total, 8 up, 8 in 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3633721115' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"} : dispatch 2024-09-17T11:53:35.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:35.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[40223]: osdmap e532: 8 total, 8 up, 8 in 2024-09-17T11:53:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:34 smithi049 ceph-mon[30017]: osdmap e532: 8 total, 8 up, 8 in 2024-09-17T11:53:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:34 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-107", "tierpool": "test-rados-api-smithi049-76013-107-cache"}]': finished 2024-09-17T11:53:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:34 smithi129 ceph-mon[33364]: osdmap e532: 8 total, 8 up, 8 in 2024-09-17T11:53:36.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:35 smithi049 ceph-mon[40223]: pgmap v806: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:36.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:35 smithi049 ceph-mon[40223]: osdmap e533: 8 total, 8 up, 8 in 2024-09-17T11:53:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:35 smithi049 ceph-mon[30017]: pgmap v806: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:35 smithi049 ceph-mon[30017]: osdmap e533: 8 total, 8 up, 8 in 2024-09-17T11:53:36.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:35 smithi129 ceph-mon[33364]: pgmap v806: 236 pgs: 236 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:53:36.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:35 smithi129 ceph-mon[33364]: osdmap e533: 8 total, 8 up, 8 in 2024-09-17T11:53:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:37.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[40223]: osdmap e534: 8 total, 8 up, 8 in 2024-09-17T11:53:37.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:37.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[30017]: osdmap e534: 8 total, 8 up, 8 in 2024-09-17T11:53:37.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:37.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:36 smithi129 ceph-mon[33364]: osdmap e534: 8 total, 8 up, 8 in 2024-09-17T11:53:37.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:37.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[40223]: pgmap v809: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[40223]: osdmap e535: 8 total, 8 up, 8 in 2024-09-17T11:53:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[30017]: pgmap v809: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[30017]: osdmap e535: 8 total, 8 up, 8 in 2024-09-17T11:53:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:38.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:37 smithi129 ceph-mon[33364]: pgmap v809: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:37 smithi129 ceph-mon[33364]: osdmap e535: 8 total, 8 up, 8 in 2024-09-17T11:53:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[40223]: osdmap e536: 8 total, 8 up, 8 in 2024-09-17T11:53:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:39.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[40223]: osdmap e537: 8 total, 8 up, 8 in 2024-09-17T11:53:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[30017]: osdmap e536: 8 total, 8 up, 8 in 2024-09-17T11:53:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:38 smithi049 ceph-mon[30017]: osdmap e537: 8 total, 8 up, 8 in 2024-09-17T11:53:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:38 smithi129 ceph-mon[33364]: osdmap e536: 8 total, 8 up, 8 in 2024-09-17T11:53:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:38 smithi129 ceph-mon[33364]: osdmap e537: 8 total, 8 up, 8 in 2024-09-17T11:53:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[40223]: pgmap v812: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[40223]: osdmap e538: 8 total, 8 up, 8 in 2024-09-17T11:53:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[30017]: pgmap v812: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[30017]: osdmap e538: 8 total, 8 up, 8 in 2024-09-17T11:53:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:39 smithi129 ceph-mon[33364]: pgmap v812: 172 pgs: 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:39 smithi129 ceph-mon[33364]: osdmap e538: 8 total, 8 up, 8 in 2024-09-17T11:53:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:53:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[40223]: pgmap v815: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[40223]: osdmap e539: 8 total, 8 up, 8 in 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[40223]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[30017]: pgmap v815: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[30017]: osdmap e539: 8 total, 8 up, 8 in 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:41 smithi049 ceph-mon[30017]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:41 smithi129 ceph-mon[33364]: pgmap v815: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:42.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:53:42.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:41 smithi129 ceph-mon[33364]: osdmap e539: 8 total, 8 up, 8 in 2024-09-17T11:53:42.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:42.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:41 smithi129 ceph-mon[33364]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[30017]: osdmap e540: 8 total, 8 up, 8 in 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[40223]: osdmap e540: 8 total, 8 up, 8 in 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:43.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:42 smithi129 ceph-mon[33364]: osdmap e540: 8 total, 8 up, 8 in 2024-09-17T11:53:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:44.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:44.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: osdmap e541: 8 total, 8 up, 8 in 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: osdmap e542: 8 total, 8 up, 8 in 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: osdmap e541: 8 total, 8 up, 8 in 2024-09-17T11:53:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: osdmap e542: 8 total, 8 up, 8 in 2024-09-17T11:53:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:44.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: pgmap v818: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-109", "overlaypool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: osdmap e541: 8 total, 8 up, 8 in 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"} : dispatch 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-109-cache", "mode": "writeback"}]': finished 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: osdmap e542: 8 total, 8 up, 8 in 2024-09-17T11:53:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:53:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[30017]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:46.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[30017]: osdmap e543: 8 total, 8 up, 8 in 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[40223]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[40223]: osdmap e543: 8 total, 8 up, 8 in 2024-09-17T11:53:46.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:45 smithi129 ceph-mon[33364]: pgmap v821: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:53:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:45 smithi129 ceph-mon[33364]: osdmap e543: 8 total, 8 up, 8 in 2024-09-17T11:53:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:53:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: osdmap e544: 8 total, 8 up, 8 in 2024-09-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[40223]: osdmap e545: 8 total, 8 up, 8 in 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: osdmap e544: 8 total, 8 up, 8 in 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:46 smithi049 ceph-mon[30017]: osdmap e545: 8 total, 8 up, 8 in 2024-09-17T11:53:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:53:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: osdmap e544: 8 total, 8 up, 8 in 2024-09-17T11:53:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:53:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:53:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:53:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:46 smithi129 ceph-mon[33364]: osdmap e545: 8 total, 8 up, 8 in 2024-09-17T11:53:48.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:53:48.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:53:48.329 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-17T11:53:48.329 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:47 smithi129 ceph-mon[33364]: osdmap e546: 8 total, 8 up, 8 in 2024-09-17T11:53:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-17T11:53:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:53:48.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[40223]: osdmap e546: 8 total, 8 up, 8 in 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: pgmap v824: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:53:48.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-17T11:53:48.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:47 smithi049 ceph-mon[30017]: osdmap e546: 8 total, 8 up, 8 in 2024-09-17T11:53:49.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:49.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:49.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:49.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:49.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:49.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:50.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[30017]: pgmap v827: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:50.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[30017]: osdmap e547: 8 total, 8 up, 8 in 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[40223]: pgmap v827: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[40223]: osdmap e547: 8 total, 8 up, 8 in 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:50.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:49 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:50.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:49 smithi129 ceph-mon[33364]: pgmap v827: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:50.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:50.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:49 smithi129 ceph-mon[33364]: osdmap e547: 8 total, 8 up, 8 in 2024-09-17T11:53:50.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"} : dispatch 2024-09-17T11:53:50.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:49 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:51.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:51.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[30017]: osdmap e548: 8 total, 8 up, 8 in 2024-09-17T11:53:51.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[30017]: osdmap e549: 8 total, 8 up, 8 in 2024-09-17T11:53:51.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:51.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[40223]: osdmap e548: 8 total, 8 up, 8 in 2024-09-17T11:53:51.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:50 smithi049 ceph-mon[40223]: osdmap e549: 8 total, 8 up, 8 in 2024-09-17T11:53:51.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-109", "tierpool": "test-rados-api-smithi049-76013-109-cache"}]': finished 2024-09-17T11:53:51.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:50 smithi129 ceph-mon[33364]: osdmap e548: 8 total, 8 up, 8 in 2024-09-17T11:53:51.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:50 smithi129 ceph-mon[33364]: osdmap e549: 8 total, 8 up, 8 in 2024-09-17T11:53:52.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[30017]: pgmap v830: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:52.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[30017]: osdmap e550: 8 total, 8 up, 8 in 2024-09-17T11:53:52.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:52.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[40223]: pgmap v830: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:52.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[40223]: osdmap e550: 8 total, 8 up, 8 in 2024-09-17T11:53:52.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:52.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:51 smithi129 ceph-mon[33364]: pgmap v830: 212 pgs: 212 active+clean; 583 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:51 smithi129 ceph-mon[33364]: osdmap e550: 8 total, 8 up, 8 in 2024-09-17T11:53:52.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:54.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[40223]: pgmap v833: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:54.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:54.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[40223]: osdmap e551: 8 total, 8 up, 8 in 2024-09-17T11:53:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[30017]: pgmap v833: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[30017]: osdmap e551: 8 total, 8 up, 8 in 2024-09-17T11:53:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:54.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:53 smithi129 ceph-mon[33364]: pgmap v833: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:54.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:54.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:53 smithi129 ceph-mon[33364]: osdmap e551: 8 total, 8 up, 8 in 2024-09-17T11:53:54.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"} : dispatch 2024-09-17T11:53:55.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:55.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[30017]: osdmap e552: 8 total, 8 up, 8 in 2024-09-17T11:53:55.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:55.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:55.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[40223]: osdmap e552: 8 total, 8 up, 8 in 2024-09-17T11:53:55.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:55 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:55.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2471510832' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-109"}]': finished 2024-09-17T11:53:55.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:55 smithi129 ceph-mon[33364]: osdmap e552: 8 total, 8 up, 8 in 2024-09-17T11:53:55.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:55 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:53:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[30017]: pgmap v836: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[30017]: osdmap e553: 8 total, 8 up, 8 in 2024-09-17T11:53:56.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[40223]: pgmap v836: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[40223]: osdmap e553: 8 total, 8 up, 8 in 2024-09-17T11:53:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:56 smithi129 ceph-mon[33364]: pgmap v836: 172 pgs: 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:53:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:56 smithi129 ceph-mon[33364]: osdmap e553: 8 total, 8 up, 8 in 2024-09-17T11:53:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:53:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:53:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:53:57.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[40223]: osdmap e554: 8 total, 8 up, 8 in 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[30017]: osdmap e554: 8 total, 8 up, 8 in 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:57.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:57.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:57 smithi129 ceph-mon[33364]: osdmap e554: 8 total, 8 up, 8 in 2024-09-17T11:53:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1663907016' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:57.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"} : dispatch 2024-09-17T11:53:58.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:58.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: osdmap e555: 8 total, 8 up, 8 in 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: osdmap e555: 8 total, 8 up, 8 in 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:58 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:58.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: pgmap v839: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi049-76013-104"}]': finished 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: osdmap e555: 8 total, 8 up, 8 in 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T11:53:58.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:58 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:53:59.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[30017]: osdmap e556: 8 total, 8 up, 8 in 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[40223]: osdmap e556: 8 total, 8 up, 8 in 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:53:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:53:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi049-76013-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T11:53:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:59 smithi129 ceph-mon[33364]: osdmap e556: 8 total, 8 up, 8 in 2024-09-17T11:53:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:53:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:53:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:00.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[30017]: pgmap v842: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[30017]: osdmap e557: 8 total, 8 up, 8 in 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[30017]: osdmap e558: 8 total, 8 up, 8 in 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[40223]: pgmap v842: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[40223]: osdmap e557: 8 total, 8 up, 8 in 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:00 smithi049 ceph-mon[40223]: osdmap e558: 8 total, 8 up, 8 in 2024-09-17T11:54:00.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:00 smithi129 ceph-mon[33364]: pgmap v842: 164 pgs: 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-17T11:54:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:00 smithi129 ceph-mon[33364]: osdmap e557: 8 total, 8 up, 8 in 2024-09-17T11:54:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:00 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi049-76013-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:00 smithi129 ceph-mon[33364]: osdmap e558: 8 total, 8 up, 8 in 2024-09-17T11:54:02.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[30017]: pgmap v845: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[30017]: osdmap e559: 8 total, 8 up, 8 in 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[40223]: pgmap v845: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[40223]: osdmap e559: 8 total, 8 up, 8 in 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:02.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:02 smithi129 ceph-mon[33364]: pgmap v845: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:02.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:02 smithi129 ceph-mon[33364]: osdmap e559: 8 total, 8 up, 8 in 2024-09-17T11:54:02.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:02.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[30017]: osdmap e560: 8 total, 8 up, 8 in 2024-09-17T11:54:03.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:03.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[40223]: osdmap e560: 8 total, 8 up, 8 in 2024-09-17T11:54:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:03.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:03.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:03 smithi129 ceph-mon[33364]: osdmap e560: 8 total, 8 up, 8 in 2024-09-17T11:54:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:03.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[40223]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[40223]: osdmap e561: 8 total, 8 up, 8 in 2024-09-17T11:54:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[30017]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[30017]: osdmap e561: 8 total, 8 up, 8 in 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:04.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:04.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:04 smithi129 ceph-mon[33364]: pgmap v848: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:04 smithi129 ceph-mon[33364]: osdmap e561: 8 total, 8 up, 8 in 2024-09-17T11:54:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:04.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:05.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:05.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[40223]: osdmap e562: 8 total, 8 up, 8 in 2024-09-17T11:54:05.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[30017]: osdmap e562: 8 total, 8 up, 8 in 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:05 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:05.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:05 smithi129 ceph-mon[33364]: osdmap e562: 8 total, 8 up, 8 in 2024-09-17T11:54:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:05 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:05.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:05 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[40223]: pgmap v851: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[40223]: osdmap e563: 8 total, 8 up, 8 in 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[30017]: pgmap v851: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[30017]: osdmap e563: 8 total, 8 up, 8 in 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:06 smithi129 ceph-mon[33364]: pgmap v851: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:06 smithi129 ceph-mon[33364]: osdmap e563: 8 total, 8 up, 8 in 2024-09-17T11:54:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"} : dispatch 2024-09-17T11:54:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:07.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:07 smithi049 ceph-mon[30017]: osdmap e564: 8 total, 8 up, 8 in 2024-09-17T11:54:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:07.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:07 smithi049 ceph-mon[40223]: osdmap e564: 8 total, 8 up, 8 in 2024-09-17T11:54:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-112"}]': finished 2024-09-17T11:54:07.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:07 smithi129 ceph-mon[33364]: osdmap e564: 8 total, 8 up, 8 in 2024-09-17T11:54:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:08 smithi049 ceph-mon[30017]: pgmap v854: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:08 smithi049 ceph-mon[30017]: osdmap e565: 8 total, 8 up, 8 in 2024-09-17T11:54:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:08 smithi049 ceph-mon[40223]: pgmap v854: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:08.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:08 smithi049 ceph-mon[40223]: osdmap e565: 8 total, 8 up, 8 in 2024-09-17T11:54:08.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:08 smithi129 ceph-mon[33364]: pgmap v854: 204 pgs: 204 active+clean; 583 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:08.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:08 smithi129 ceph-mon[33364]: osdmap e565: 8 total, 8 up, 8 in 2024-09-17T11:54:08.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[40223]: osdmap e566: 8 total, 8 up, 8 in 2024-09-17T11:54:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[30017]: osdmap e566: 8 total, 8 up, 8 in 2024-09-17T11:54:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:09 smithi129 ceph-mon[33364]: osdmap e566: 8 total, 8 up, 8 in 2024-09-17T11:54:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:09.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: osdmap e567: 8 total, 8 up, 8 in 2024-09-17T11:54:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: osdmap e567: 8 total, 8 up, 8 in 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:10 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:10.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: pgmap v857: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: osdmap e567: 8 total, 8 up, 8 in 2024-09-17T11:54:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:10.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:10 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[40223]: osdmap e568: 8 total, 8 up, 8 in 2024-09-17T11:54:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:11.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[30017]: osdmap e568: 8 total, 8 up, 8 in 2024-09-17T11:54:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:11.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:11.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:11 smithi129 ceph-mon[33364]: osdmap e568: 8 total, 8 up, 8 in 2024-09-17T11:54:11.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:11.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[40223]: pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[40223]: osdmap e569: 8 total, 8 up, 8 in 2024-09-17T11:54:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[30017]: pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[30017]: osdmap e569: 8 total, 8 up, 8 in 2024-09-17T11:54:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:12 smithi129 ceph-mon[33364]: pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-17T11:54:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:12.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:12 smithi129 ceph-mon[33364]: osdmap e569: 8 total, 8 up, 8 in 2024-09-17T11:54:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:12.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"} : dispatch 2024-09-17T11:54:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"}]': finished 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: osdmap e570: 8 total, 8 up, 8 in 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"}]': finished 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: osdmap e570: 8 total, 8 up, 8 in 2024-09-17T11:54:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-114", "mode": "writeback"}]': finished 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: osdmap e570: 8 total, 8 up, 8 in 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:13.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:14.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: pgmap v863: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:14.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: osdmap e571: 8 total, 8 up, 8 in 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[30017]: osdmap e572: 8 total, 8 up, 8 in 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: pgmap v863: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: osdmap e571: 8 total, 8 up, 8 in 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:14.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:14.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:14 smithi049 ceph-mon[40223]: osdmap e572: 8 total, 8 up, 8 in 2024-09-17T11:54:14.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: pgmap v863: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: osdmap e571: 8 total, 8 up, 8 in 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"} : dispatch 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-114"}]': finished 2024-09-17T11:54:14.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:14 smithi129 ceph-mon[33364]: osdmap e572: 8 total, 8 up, 8 in 2024-09-17T11:54:15.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:15 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:15.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:15 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:15 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:16.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:16 smithi049 ceph-mon[40223]: pgmap v866: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:16.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:16 smithi049 ceph-mon[40223]: osdmap e573: 8 total, 8 up, 8 in 2024-09-17T11:54:16.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:16 smithi049 ceph-mon[30017]: pgmap v866: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:16.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:16 smithi049 ceph-mon[30017]: osdmap e573: 8 total, 8 up, 8 in 2024-09-17T11:54:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:16 smithi129 ceph-mon[33364]: pgmap v866: 204 pgs: 204 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-17T11:54:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:16 smithi129 ceph-mon[33364]: osdmap e573: 8 total, 8 up, 8 in 2024-09-17T11:54:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[30017]: osdmap e574: 8 total, 8 up, 8 in 2024-09-17T11:54:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:17.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[40223]: osdmap e574: 8 total, 8 up, 8 in 2024-09-17T11:54:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:17 smithi129 ceph-mon[33364]: osdmap e574: 8 total, 8 up, 8 in 2024-09-17T11:54:17.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:17.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:18.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:18.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:18 smithi129 ceph-mon[33364]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:18.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:18.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:18 smithi129 ceph-mon[33364]: osdmap e575: 8 total, 8 up, 8 in 2024-09-17T11:54:18.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:18 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[40223]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[40223]: osdmap e575: 8 total, 8 up, 8 in 2024-09-17T11:54:18.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[30017]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[30017]: osdmap e575: 8 total, 8 up, 8 in 2024-09-17T11:54:18.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:18 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:19 smithi129 ceph-mon[33364]: osdmap e576: 8 total, 8 up, 8 in 2024-09-17T11:54:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:19 smithi129 ceph-mon[33364]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:54:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[40223]: osdmap e576: 8 total, 8 up, 8 in 2024-09-17T11:54:19.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[40223]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[30017]: osdmap e576: 8 total, 8 up, 8 in 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:19.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:19 smithi049 ceph-mon[30017]: pgmap v872: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-09-17T11:54:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:20 smithi129 ceph-mon[33364]: osdmap e577: 8 total, 8 up, 8 in 2024-09-17T11:54:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[40223]: osdmap e577: 8 total, 8 up, 8 in 2024-09-17T11:54:20.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:20.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[30017]: osdmap e577: 8 total, 8 up, 8 in 2024-09-17T11:54:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:20.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:21 smithi129 ceph-mon[33364]: osdmap e578: 8 total, 8 up, 8 in 2024-09-17T11:54:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:21 smithi129 ceph-mon[33364]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:54:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[30017]: osdmap e578: 8 total, 8 up, 8 in 2024-09-17T11:54:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[30017]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[40223]: osdmap e578: 8 total, 8 up, 8 in 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"} : dispatch 2024-09-17T11:54:21.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:21 smithi049 ceph-mon[40223]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:54:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:22 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"}]': finished 2024-09-17T11:54:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:22 smithi129 ceph-mon[33364]: osdmap e579: 8 total, 8 up, 8 in 2024-09-17T11:54:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-17T11:54:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:22 smithi129 ceph-mon[33364]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-17T11:54:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"}]': finished 2024-09-17T11:54:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[40223]: osdmap e579: 8 total, 8 up, 8 in 2024-09-17T11:54:22.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[40223]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-116", "mode": "writeback"}]': finished 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[30017]: osdmap e579: 8 total, 8 up, 8 in 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-17T11:54:22.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:22 smithi049 ceph-mon[30017]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-17T11:54:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[30017]: 325.4 scrub starts 2024-09-17T11:54:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[30017]: 325.4 scrub ok 2024-09-17T11:54:23.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[30017]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 679 B/s wr, 2 op/s 2024-09-17T11:54:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[40223]: 325.4 scrub starts 2024-09-17T11:54:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[40223]: 325.4 scrub ok 2024-09-17T11:54:23.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:23 smithi049 ceph-mon[40223]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 679 B/s wr, 2 op/s 2024-09-17T11:54:23.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:23 smithi129 ceph-mon[33364]: 325.4 scrub starts 2024-09-17T11:54:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:23 smithi129 ceph-mon[33364]: 325.4 scrub ok 2024-09-17T11:54:23.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:23 smithi129 ceph-mon[33364]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 679 B/s wr, 2 op/s 2024-09-17T11:54:26.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:25 smithi049 ceph-mon[30017]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:54:26.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:25 smithi049 ceph-mon[40223]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:54:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:25 smithi129 ceph-mon[33364]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:54:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:28.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:27 smithi129 ceph-mon[33364]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 409 B/s wr, 1 op/s 2024-09-17T11:54:28.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:27 smithi049 ceph-mon[40223]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 409 B/s wr, 1 op/s 2024-09-17T11:54:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:27 smithi049 ceph-mon[30017]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 409 B/s wr, 1 op/s 2024-09-17T11:54:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:28.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:28 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:29.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:28 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:28 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:29 smithi049 ceph-mon[40223]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 361 B/s wr, 2 op/s 2024-09-17T11:54:30.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:29 smithi049 ceph-mon[30017]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 361 B/s wr, 2 op/s 2024-09-17T11:54:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:29 smithi129 ceph-mon[33364]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 361 B/s wr, 2 op/s 2024-09-17T11:54:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:31 smithi049 ceph-mon[40223]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 307 B/s wr, 1 op/s 2024-09-17T11:54:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:31 smithi049 ceph-mon[30017]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 307 B/s wr, 1 op/s 2024-09-17T11:54:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:31 smithi129 ceph-mon[33364]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 307 B/s wr, 1 op/s 2024-09-17T11:54:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:33 smithi049 ceph-mon[30017]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 267 B/s wr, 2 op/s 2024-09-17T11:54:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:33 smithi049 ceph-mon[40223]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 267 B/s wr, 2 op/s 2024-09-17T11:54:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:33 smithi129 ceph-mon[33364]: pgmap v882: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 267 B/s wr, 2 op/s 2024-09-17T11:54:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:35 smithi049 ceph-mon[40223]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:54:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:35 smithi049 ceph-mon[30017]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:54:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:35 smithi129 ceph-mon[33364]: pgmap v883: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:54:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:38.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:37 smithi129 ceph-mon[33364]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:54:38.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:37 smithi129 ceph-mon[33364]: osdmap e580: 8 total, 8 up, 8 in 2024-09-17T11:54:38.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.132 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[40223]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:54:38.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[40223]: osdmap e580: 8 total, 8 up, 8 in 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[30017]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[30017]: osdmap e580: 8 total, 8 up, 8 in 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:38.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: osdmap e581: 8 total, 8 up, 8 in 2024-09-17T11:54:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: osdmap e581: 8 total, 8 up, 8 in 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: osdmap e581: 8 total, 8 up, 8 in 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"} : dispatch 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-116"}]': finished 2024-09-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[40223]: pgmap v887: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[40223]: osdmap e582: 8 total, 8 up, 8 in 2024-09-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[40223]: osdmap e583: 8 total, 8 up, 8 in 2024-09-17T11:54:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[30017]: pgmap v887: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:54:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[30017]: osdmap e582: 8 total, 8 up, 8 in 2024-09-17T11:54:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:39 smithi049 ceph-mon[30017]: osdmap e583: 8 total, 8 up, 8 in 2024-09-17T11:54:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:39 smithi129 ceph-mon[33364]: pgmap v887: 204 pgs: 204 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:54:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:39 smithi129 ceph-mon[33364]: osdmap e582: 8 total, 8 up, 8 in 2024-09-17T11:54:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:39 smithi129 ceph-mon[33364]: osdmap e583: 8 total, 8 up, 8 in 2024-09-17T11:54:41.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:41 smithi129 ceph-mon[33364]: osdmap e584: 8 total, 8 up, 8 in 2024-09-17T11:54:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:41 smithi129 ceph-mon[33364]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 782 B/s rd, 1 op/s 2024-09-17T11:54:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[30017]: osdmap e584: 8 total, 8 up, 8 in 2024-09-17T11:54:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[30017]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 782 B/s rd, 1 op/s 2024-09-17T11:54:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[40223]: osdmap e584: 8 total, 8 up, 8 in 2024-09-17T11:54:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:41.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:41 smithi049 ceph-mon[40223]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 782 B/s rd, 1 op/s 2024-09-17T11:54:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:42 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:42.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:42 smithi129 ceph-mon[33364]: osdmap e585: 8 total, 8 up, 8 in 2024-09-17T11:54:42.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:42.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:42.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[30017]: osdmap e585: 8 total, 8 up, 8 in 2024-09-17T11:54:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:42.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:42 smithi049 ceph-mon[40223]: osdmap e585: 8 total, 8 up, 8 in 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: osdmap e586: 8 total, 8 up, 8 in 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:43.421 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[30017]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:43.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: osdmap e586: 8 total, 8 up, 8 in 2024-09-17T11:54:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:43.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:43 smithi129 ceph-mon[33364]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: osdmap e586: 8 total, 8 up, 8 in 2024-09-17T11:54:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:43.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:43.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:43.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:43 smithi049 ceph-mon[40223]: pgmap v894: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:44.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: osdmap e587: 8 total, 8 up, 8 in 2024-09-17T11:54:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:44.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:44 smithi129 ceph-mon[33364]: osdmap e588: 8 total, 8 up, 8 in 2024-09-17T11:54:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: osdmap e587: 8 total, 8 up, 8 in 2024-09-17T11:54:44.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[30017]: osdmap e588: 8 total, 8 up, 8 in 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: osdmap e587: 8 total, 8 up, 8 in 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:44 smithi049 ceph-mon[40223]: osdmap e588: 8 total, 8 up, 8 in 2024-09-17T11:54:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:45.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"}]': finished 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[40223]: osdmap e589: 8 total, 8 up, 8 in 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"}]': finished 2024-09-17T11:54:45.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:45 smithi049 ceph-mon[30017]: osdmap e589: 8 total, 8 up, 8 in 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"} : dispatch 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-118", "mode": "writeback"}]': finished 2024-09-17T11:54:45.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:45 smithi129 ceph-mon[33364]: osdmap e589: 8 total, 8 up, 8 in 2024-09-17T11:54:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:47.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[40223]: osdmap e590: 8 total, 8 up, 8 in 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[40223]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[30017]: osdmap e590: 8 total, 8 up, 8 in 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:47 smithi049 ceph-mon[30017]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:47.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:47 smithi129 ceph-mon[33364]: osdmap e590: 8 total, 8 up, 8 in 2024-09-17T11:54:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:47.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:47 smithi129 ceph-mon[33364]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:48.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:48.822 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:48.823 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: osdmap e591: 8 total, 8 up, 8 in 2024-09-17T11:54:48.823 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.823 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.823 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:48.823 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:54:48.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:48.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: osdmap e591: 8 total, 8 up, 8 in 2024-09-17T11:54:48.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.824 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:48.825 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:48 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: osdmap e591: 8 total, 8 up, 8 in 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"} : dispatch 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:54:48.831 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:48 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:54:49.801 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:49 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:49.802 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:49.802 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:49 smithi129 ceph-mon[33364]: osdmap e592: 8 total, 8 up, 8 in 2024-09-17T11:54:49.802 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:49 smithi129 ceph-mon[33364]: pgmap v903: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:54:49.802 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:49 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[40223]: osdmap e592: 8 total, 8 up, 8 in 2024-09-17T11:54:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[40223]: pgmap v903: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:54:49.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-118"}]': finished 2024-09-17T11:54:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[30017]: osdmap e592: 8 total, 8 up, 8 in 2024-09-17T11:54:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[30017]: pgmap v903: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:54:49.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:49 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:50 smithi049 ceph-mon[40223]: osdmap e593: 8 total, 8 up, 8 in 2024-09-17T11:54:50.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:50 smithi049 ceph-mon[30017]: osdmap e593: 8 total, 8 up, 8 in 2024-09-17T11:54:50.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:50 smithi129 ceph-mon[33364]: osdmap e593: 8 total, 8 up, 8 in 2024-09-17T11:54:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: osdmap e594: 8 total, 8 up, 8 in 2024-09-17T11:54:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:51 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:51.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: osdmap e594: 8 total, 8 up, 8 in 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: osdmap e594: 8 total, 8 up, 8 in 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:51.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:51 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:52.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[30017]: osdmap e595: 8 total, 8 up, 8 in 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[40223]: osdmap e595: 8 total, 8 up, 8 in 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:52.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:54:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:52 smithi129 ceph-mon[33364]: osdmap e595: 8 total, 8 up, 8 in 2024-09-17T11:54:52.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:52.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:54:53.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: osdmap e596: 8 total, 8 up, 8 in 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: osdmap e597: 8 total, 8 up, 8 in 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:53.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: osdmap e596: 8 total, 8 up, 8 in 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:54:53.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:54:53.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:53.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: osdmap e597: 8 total, 8 up, 8 in 2024-09-17T11:54:53.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:53.903 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:54:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: osdmap e596: 8 total, 8 up, 8 in 2024-09-17T11:54:53.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: pgmap v909: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: osdmap e597: 8 total, 8 up, 8 in 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:53.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"} : dispatch 2024-09-17T11:54:54.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:54 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"}]': finished 2024-09-17T11:54:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:54 smithi129 ceph-mon[33364]: osdmap e598: 8 total, 8 up, 8 in 2024-09-17T11:54:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:54.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"}]': finished 2024-09-17T11:54:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[40223]: osdmap e598: 8 total, 8 up, 8 in 2024-09-17T11:54:55.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-120", "mode": "writeback"}]': finished 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[30017]: osdmap e598: 8 total, 8 up, 8 in 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:54:55.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:55 smithi129 ceph-mon[33364]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:54:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:55 smithi129 ceph-mon[33364]: osdmap e599: 8 total, 8 up, 8 in 2024-09-17T11:54:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:55.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[40223]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:54:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[40223]: osdmap e599: 8 total, 8 up, 8 in 2024-09-17T11:54:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[30017]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[30017]: osdmap e599: 8 total, 8 up, 8 in 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"} : dispatch 2024-09-17T11:54:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[30017]: osdmap e600: 8 total, 8 up, 8 in 2024-09-17T11:54:56.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:54:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:54:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:56 smithi049 ceph-mon[40223]: osdmap e600: 8 total, 8 up, 8 in 2024-09-17T11:54:56.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:56 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:54:56.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-120"}]': finished 2024-09-17T11:54:56.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:56 smithi129 ceph-mon[33364]: osdmap e600: 8 total, 8 up, 8 in 2024-09-17T11:54:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[40223]: pgmap v915: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[40223]: osdmap e601: 8 total, 8 up, 8 in 2024-09-17T11:54:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[30017]: pgmap v915: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[30017]: osdmap e601: 8 total, 8 up, 8 in 2024-09-17T11:54:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:58.154 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:57 smithi129 ceph-mon[33364]: pgmap v915: 204 pgs: 204 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:54:58.154 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:57 smithi129 ceph-mon[33364]: osdmap e601: 8 total, 8 up, 8 in 2024-09-17T11:54:58.154 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:54:58.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:54:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:54:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[40223]: osdmap e602: 8 total, 8 up, 8 in 2024-09-17T11:54:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[30017]: osdmap e602: 8 total, 8 up, 8 in 2024-09-17T11:54:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:58 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:54:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:58 smithi129 ceph-mon[33364]: osdmap e602: 8 total, 8 up, 8 in 2024-09-17T11:54:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:54:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: pgmap v918: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: osdmap e603: 8 total, 8 up, 8 in 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: pgmap v918: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: osdmap e603: 8 total, 8 up, 8 in 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:54:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: pgmap v918: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: osdmap e603: 8 total, 8 up, 8 in 2024-09-17T11:55:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:54:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[40223]: osdmap e604: 8 total, 8 up, 8 in 2024-09-17T11:55:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[40223]: pgmap v921: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[30017]: osdmap e604: 8 total, 8 up, 8 in 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:01 smithi049 ceph-mon[30017]: pgmap v921: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:01.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:01 smithi129 ceph-mon[33364]: osdmap e604: 8 total, 8 up, 8 in 2024-09-17T11:55:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:01.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:01 smithi129 ceph-mon[33364]: pgmap v921: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:02.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:02.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[40223]: osdmap e605: 8 total, 8 up, 8 in 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[30017]: osdmap e605: 8 total, 8 up, 8 in 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:02 smithi129 ceph-mon[33364]: osdmap e605: 8 total, 8 up, 8 in 2024-09-17T11:55:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"} : dispatch 2024-09-17T11:55:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"}]': finished 2024-09-17T11:55:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: osdmap e606: 8 total, 8 up, 8 in 2024-09-17T11:55:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[40223]: pgmap v924: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"}]': finished 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: osdmap e606: 8 total, 8 up, 8 in 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:03 smithi049 ceph-mon[30017]: pgmap v924: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:03.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-122", "mode": "writeback"}]': finished 2024-09-17T11:55:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: osdmap e606: 8 total, 8 up, 8 in 2024-09-17T11:55:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:03 smithi129 ceph-mon[33364]: pgmap v924: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:04.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[40223]: osdmap e607: 8 total, 8 up, 8 in 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[30017]: osdmap e607: 8 total, 8 up, 8 in 2024-09-17T11:55:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:04 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:04.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:04 smithi129 ceph-mon[33364]: osdmap e607: 8 total, 8 up, 8 in 2024-09-17T11:55:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:04 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"} : dispatch 2024-09-17T11:55:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:04 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[40223]: osdmap e608: 8 total, 8 up, 8 in 2024-09-17T11:55:05.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[40223]: pgmap v927: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[30017]: osdmap e608: 8 total, 8 up, 8 in 2024-09-17T11:55:05.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:05 smithi049 ceph-mon[30017]: pgmap v927: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:05 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:05 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-122"}]': finished 2024-09-17T11:55:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:05 smithi129 ceph-mon[33364]: osdmap e608: 8 total, 8 up, 8 in 2024-09-17T11:55:05.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:05 smithi129 ceph-mon[33364]: pgmap v927: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-17T11:55:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:06 smithi049 ceph-mon[40223]: osdmap e609: 8 total, 8 up, 8 in 2024-09-17T11:55:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:06 smithi049 ceph-mon[30017]: osdmap e609: 8 total, 8 up, 8 in 2024-09-17T11:55:06.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:06 smithi129 ceph-mon[33364]: osdmap e609: 8 total, 8 up, 8 in 2024-09-17T11:55:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[40223]: osdmap e610: 8 total, 8 up, 8 in 2024-09-17T11:55:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[40223]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[30017]: osdmap e610: 8 total, 8 up, 8 in 2024-09-17T11:55:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:07 smithi049 ceph-mon[30017]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:07.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:07 smithi129 ceph-mon[33364]: osdmap e610: 8 total, 8 up, 8 in 2024-09-17T11:55:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:07 smithi129 ceph-mon[33364]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:08.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:08 smithi049 ceph-mon[40223]: osdmap e611: 8 total, 8 up, 8 in 2024-09-17T11:55:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:08 smithi049 ceph-mon[30017]: osdmap e611: 8 total, 8 up, 8 in 2024-09-17T11:55:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:08.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:08 smithi129 ceph-mon[33364]: osdmap e611: 8 total, 8 up, 8 in 2024-09-17T11:55:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: osdmap e612: 8 total, 8 up, 8 in 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: osdmap e612: 8 total, 8 up, 8 in 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:09 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: osdmap e612: 8 total, 8 up, 8 in 2024-09-17T11:55:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: pgmap v933: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:09 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[40223]: osdmap e613: 8 total, 8 up, 8 in 2024-09-17T11:55:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:10.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[30017]: osdmap e613: 8 total, 8 up, 8 in 2024-09-17T11:55:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:10 smithi129 ceph-mon[33364]: osdmap e613: 8 total, 8 up, 8 in 2024-09-17T11:55:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:10.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[30017]: osdmap e614: 8 total, 8 up, 8 in 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[30017]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[40223]: osdmap e614: 8 total, 8 up, 8 in 2024-09-17T11:55:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:11 smithi049 ceph-mon[40223]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:55:11.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:11.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:11 smithi129 ceph-mon[33364]: osdmap e614: 8 total, 8 up, 8 in 2024-09-17T11:55:11.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"} : dispatch 2024-09-17T11:55:11.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:11 smithi129 ceph-mon[33364]: pgmap v936: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:55:12.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:12.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"}]': finished 2024-09-17T11:55:12.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[30017]: osdmap e615: 8 total, 8 up, 8 in 2024-09-17T11:55:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"}]': finished 2024-09-17T11:55:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:12 smithi049 ceph-mon[40223]: osdmap e615: 8 total, 8 up, 8 in 2024-09-17T11:55:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:12 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-124", "mode": "writeback"}]': finished 2024-09-17T11:55:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:12 smithi129 ceph-mon[33364]: osdmap e615: 8 total, 8 up, 8 in 2024-09-17T11:55:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: osdmap e616: 8 total, 8 up, 8 in 2024-09-17T11:55:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[30017]: pgmap v939: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: osdmap e616: 8 total, 8 up, 8 in 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:13 smithi049 ceph-mon[40223]: pgmap v939: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: osdmap e616: 8 total, 8 up, 8 in 2024-09-17T11:55:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:13 smithi129 ceph-mon[33364]: pgmap v939: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: osdmap e617: 8 total, 8 up, 8 in 2024-09-17T11:55:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[40223]: osdmap e618: 8 total, 8 up, 8 in 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: osdmap e617: 8 total, 8 up, 8 in 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:14.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:14 smithi049 ceph-mon[30017]: osdmap e618: 8 total, 8 up, 8 in 2024-09-17T11:55:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: osdmap e617: 8 total, 8 up, 8 in 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"} : dispatch 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-124"}]': finished 2024-09-17T11:55:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:14 smithi129 ceph-mon[33364]: osdmap e618: 8 total, 8 up, 8 in 2024-09-17T11:55:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:15 smithi049 ceph-mon[30017]: pgmap v942: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:15.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:15 smithi049 ceph-mon[30017]: osdmap e619: 8 total, 8 up, 8 in 2024-09-17T11:55:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:15 smithi049 ceph-mon[40223]: pgmap v942: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:15 smithi049 ceph-mon[40223]: osdmap e619: 8 total, 8 up, 8 in 2024-09-17T11:55:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:15 smithi129 ceph-mon[33364]: pgmap v942: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-17T11:55:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:15 smithi129 ceph-mon[33364]: osdmap e619: 8 total, 8 up, 8 in 2024-09-17T11:55:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:17.817 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[40223]: osdmap e620: 8 total, 8 up, 8 in 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[40223]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[30017]: osdmap e620: 8 total, 8 up, 8 in 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.818 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:17 smithi049 ceph-mon[30017]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:17.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:17 smithi129 ceph-mon[33364]: osdmap e620: 8 total, 8 up, 8 in 2024-09-17T11:55:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:17 smithi129 ceph-mon[33364]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:18.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[40223]: osdmap e621: 8 total, 8 up, 8 in 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[30017]: osdmap e621: 8 total, 8 up, 8 in 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:18 smithi129 ceph-mon[33364]: osdmap e621: 8 total, 8 up, 8 in 2024-09-17T11:55:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: osdmap e622: 8 total, 8 up, 8 in 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: osdmap e622: 8 total, 8 up, 8 in 2024-09-17T11:55:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:19.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:19 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: osdmap e622: 8 total, 8 up, 8 in 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:19 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:20.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[40223]: osdmap e623: 8 total, 8 up, 8 in 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[30017]: osdmap e623: 8 total, 8 up, 8 in 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:20 smithi129 ceph-mon[33364]: osdmap e623: 8 total, 8 up, 8 in 2024-09-17T11:55:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"} : dispatch 2024-09-17T11:55:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"}]': finished 2024-09-17T11:55:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: osdmap e624: 8 total, 8 up, 8 in 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[40223]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"}]': finished 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: osdmap e624: 8 total, 8 up, 8 in 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:21 smithi049 ceph-mon[30017]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-126", "mode": "writeback"}]': finished 2024-09-17T11:55:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: osdmap e624: 8 total, 8 up, 8 in 2024-09-17T11:55:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:21 smithi129 ceph-mon[33364]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[40223]: osdmap e625: 8 total, 8 up, 8 in 2024-09-17T11:55:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:22.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[30017]: osdmap e625: 8 total, 8 up, 8 in 2024-09-17T11:55:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:22.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:22.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:22 smithi129 ceph-mon[33364]: osdmap e625: 8 total, 8 up, 8 in 2024-09-17T11:55:22.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:22.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"} : dispatch 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9065 ms) 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6043 ms) 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5027 ms) 2024-09-17T11:55:23.472 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25437 ms) 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (13916 ms) 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12086 ms) 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 00000012 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12074 ms) 2024-09-17T11:55:23.473 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6053 ms) 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (557565 ms total) 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1010 ms) 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11060 ms) 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18053 ms) 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-17T11:55:23.474 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30126 ms total) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7044 ms) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8050 ms) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: waiting for scrub... 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: done waiting 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24266 ms) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10048 ms) 2024-09-17T11:55:23.475 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-17T11:55:23.476 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7968 ms) 2024-09-17T11:55:23.476 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-17T11:55:23.476 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8039 ms) 2024-09-17T11:55:23.476 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-17T11:55:23.476 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (9966 ms) 2024-09-17T11:55:23.477 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-17T11:55:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[40223]: osdmap e626: 8 total, 8 up, 8 in 2024-09-17T11:55:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[40223]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:55:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[30017]: osdmap e626: 8 total, 8 up, 8 in 2024-09-17T11:55:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:23 smithi049 ceph-mon[30017]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:55:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:23 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-126"}]': finished 2024-09-17T11:55:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:23 smithi129 ceph-mon[33364]: osdmap e626: 8 total, 8 up, 8 in 2024-09-17T11:55:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:23 smithi129 ceph-mon[33364]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-17T11:55:24.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:24 smithi049 ceph-mon[40223]: osdmap e627: 8 total, 8 up, 8 in 2024-09-17T11:55:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:24 smithi049 ceph-mon[30017]: osdmap e627: 8 total, 8 up, 8 in 2024-09-17T11:55:24.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:24 smithi129 ceph-mon[33364]: osdmap e627: 8 total, 8 up, 8 in 2024-09-17T11:55:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[40223]: osdmap e628: 8 total, 8 up, 8 in 2024-09-17T11:55:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[40223]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[30017]: osdmap e628: 8 total, 8 up, 8 in 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:25 smithi049 ceph-mon[30017]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:25.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:25 smithi129 ceph-mon[33364]: osdmap e628: 8 total, 8 up, 8 in 2024-09-17T11:55:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:25 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:25.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:25 smithi129 ceph-mon[33364]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[40223]: osdmap e629: 8 total, 8 up, 8 in 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[30017]: osdmap e629: 8 total, 8 up, 8 in 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:26.902 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:26.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:26.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:26 smithi129 ceph-mon[33364]: osdmap e629: 8 total, 8 up, 8 in 2024-09-17T11:55:26.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:26.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: osdmap e630: 8 total, 8 up, 8 in 2024-09-17T11:55:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[30017]: osdmap e631: 8 total, 8 up, 8 in 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: osdmap e630: 8 total, 8 up, 8 in 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:27.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:27.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:27 smithi049 ceph-mon[40223]: osdmap e631: 8 total, 8 up, 8 in 2024-09-17T11:55:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: osdmap e630: 8 total, 8 up, 8 in 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:27 smithi129 ceph-mon[33364]: osdmap e631: 8 total, 8 up, 8 in 2024-09-17T11:55:28.496 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:28.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"}]': finished 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[40223]: osdmap e632: 8 total, 8 up, 8 in 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:28.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:28.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:28.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"}]': finished 2024-09-17T11:55:28.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:28 smithi049 ceph-mon[30017]: osdmap e632: 8 total, 8 up, 8 in 2024-09-17T11:55:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"} : dispatch 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-128", "mode": "writeback"}]': finished 2024-09-17T11:55:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:28 smithi129 ceph-mon[33364]: osdmap e632: 8 total, 8 up, 8 in 2024-09-17T11:55:29.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:29.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:29 smithi049 ceph-mon[30017]: pgmap v963: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:29.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:29.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:29 smithi049 ceph-mon[40223]: pgmap v963: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:29 smithi129 ceph-mon[33364]: pgmap v963: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:31 smithi049 ceph-mon[40223]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 0 op/s 2024-09-17T11:55:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:31 smithi049 ceph-mon[30017]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 0 op/s 2024-09-17T11:55:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:31 smithi129 ceph-mon[33364]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 0 op/s 2024-09-17T11:55:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[40223]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 329 B/s wr, 2 op/s 2024-09-17T11:55:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[30017]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 329 B/s wr, 2 op/s 2024-09-17T11:55:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:33 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:33 smithi129 ceph-mon[33364]: pgmap v965: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 329 B/s wr, 2 op/s 2024-09-17T11:55:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:33 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:34.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:33 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: osdmap e633: 8 total, 8 up, 8 in 2024-09-17T11:55:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[40223]: osdmap e634: 8 total, 8 up, 8 in 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: osdmap e633: 8 total, 8 up, 8 in 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:35.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:34 smithi049 ceph-mon[30017]: osdmap e634: 8 total, 8 up, 8 in 2024-09-17T11:55:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: osdmap e633: 8 total, 8 up, 8 in 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"} : dispatch 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-128"}]': finished 2024-09-17T11:55:35.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:34 smithi129 ceph-mon[33364]: osdmap e634: 8 total, 8 up, 8 in 2024-09-17T11:55:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:35 smithi049 ceph-mon[30017]: pgmap v967: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 284 B/s wr, 1 op/s 2024-09-17T11:55:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:35 smithi049 ceph-mon[40223]: pgmap v967: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 284 B/s wr, 1 op/s 2024-09-17T11:55:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:35 smithi129 ceph-mon[33364]: pgmap v967: 204 pgs: 204 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 284 B/s wr, 1 op/s 2024-09-17T11:55:36.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:36 smithi049 ceph-mon[30017]: osdmap e635: 8 total, 8 up, 8 in 2024-09-17T11:55:36.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:36 smithi049 ceph-mon[40223]: osdmap e635: 8 total, 8 up, 8 in 2024-09-17T11:55:36.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:37.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:36 smithi129 ceph-mon[33364]: osdmap e635: 8 total, 8 up, 8 in 2024-09-17T11:55:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[30017]: pgmap v970: 172 pgs: 172 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:55:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[30017]: osdmap e636: 8 total, 8 up, 8 in 2024-09-17T11:55:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[40223]: pgmap v970: 172 pgs: 172 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:55:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[40223]: osdmap e636: 8 total, 8 up, 8 in 2024-09-17T11:55:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:37 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:37 smithi129 ceph-mon[33364]: pgmap v970: 172 pgs: 172 active+clean; 583 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:55:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:37 smithi129 ceph-mon[33364]: osdmap e636: 8 total, 8 up, 8 in 2024-09-17T11:55:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:37 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:38.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[40223]: osdmap e637: 8 total, 8 up, 8 in 2024-09-17T11:55:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:39.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[30017]: osdmap e637: 8 total, 8 up, 8 in 2024-09-17T11:55:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:39.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:39.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:38 smithi129 ceph-mon[33364]: osdmap e637: 8 total, 8 up, 8 in 2024-09-17T11:55:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:39.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: pgmap v973: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: osdmap e638: 8 total, 8 up, 8 in 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: pgmap v973: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:40.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: osdmap e638: 8 total, 8 up, 8 in 2024-09-17T11:55:40.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:40.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: pgmap v973: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:55:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: osdmap e638: 8 total, 8 up, 8 in 2024-09-17T11:55:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:41.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:41.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: osdmap e639: 8 total, 8 up, 8 in 2024-09-17T11:55:41.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"}]': finished 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: osdmap e639: 8 total, 8 up, 8 in 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:41.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"}]': finished 2024-09-17T11:55:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: osdmap e639: 8 total, 8 up, 8 in 2024-09-17T11:55:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"} : dispatch 2024-09-17T11:55:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-130", "mode": "writeback"}]': finished 2024-09-17T11:55:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[40223]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[40223]: osdmap e640: 8 total, 8 up, 8 in 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[30017]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[30017]: osdmap e640: 8 total, 8 up, 8 in 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:41 smithi129 ceph-mon[33364]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:41 smithi129 ceph-mon[33364]: osdmap e640: 8 total, 8 up, 8 in 2024-09-17T11:55:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: osdmap e641: 8 total, 8 up, 8 in 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: osdmap e641: 8 total, 8 up, 8 in 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:43.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: osdmap e641: 8 total, 8 up, 8 in 2024-09-17T11:55:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"} : dispatch 2024-09-17T11:55:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[40223]: pgmap v979: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-17T11:55:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[40223]: osdmap e642: 8 total, 8 up, 8 in 2024-09-17T11:55:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[30017]: pgmap v979: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-17T11:55:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:43 smithi049 ceph-mon[30017]: osdmap e642: 8 total, 8 up, 8 in 2024-09-17T11:55:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:43 smithi129 ceph-mon[33364]: pgmap v979: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-17T11:55:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:43 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-130"}]': finished 2024-09-17T11:55:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:43 smithi129 ceph-mon[33364]: osdmap e642: 8 total, 8 up, 8 in 2024-09-17T11:55:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:44 smithi049 ceph-mon[40223]: osdmap e643: 8 total, 8 up, 8 in 2024-09-17T11:55:45.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:44 smithi049 ceph-mon[30017]: osdmap e643: 8 total, 8 up, 8 in 2024-09-17T11:55:45.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:44 smithi129 ceph-mon[33364]: osdmap e643: 8 total, 8 up, 8 in 2024-09-17T11:55:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[40223]: pgmap v982: 172 pgs: 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[40223]: osdmap e644: 8 total, 8 up, 8 in 2024-09-17T11:55:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[30017]: pgmap v982: 172 pgs: 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[30017]: osdmap e644: 8 total, 8 up, 8 in 2024-09-17T11:55:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:45 smithi129 ceph-mon[33364]: pgmap v982: 172 pgs: 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-17T11:55:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:45 smithi129 ceph-mon[33364]: osdmap e644: 8 total, 8 up, 8 in 2024-09-17T11:55:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:46.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:46.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[30017]: osdmap e645: 8 total, 8 up, 8 in 2024-09-17T11:55:46.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[40223]: osdmap e645: 8 total, 8 up, 8 in 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:55:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:46 smithi129 ceph-mon[33364]: osdmap e645: 8 total, 8 up, 8 in 2024-09-17T11:55:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:55:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[30017]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[30017]: osdmap e646: 8 total, 8 up, 8 in 2024-09-17T11:55:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[40223]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[40223]: osdmap e646: 8 total, 8 up, 8 in 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:47 smithi129 ceph-mon[33364]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:55:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:47 smithi129 ceph-mon[33364]: osdmap e646: 8 total, 8 up, 8 in 2024-09-17T11:55:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:48.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[40223]: osdmap e647: 8 total, 8 up, 8 in 2024-09-17T11:55:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[30017]: osdmap e647: 8 total, 8 up, 8 in 2024-09-17T11:55:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:48 smithi129 ceph-mon[33364]: osdmap e647: 8 total, 8 up, 8 in 2024-09-17T11:55:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"} : dispatch 2024-09-17T11:55:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[40223]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"}]': finished 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[40223]: osdmap e648: 8 total, 8 up, 8 in 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[30017]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"}]': finished 2024-09-17T11:55:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:49 smithi049 ceph-mon[30017]: osdmap e648: 8 total, 8 up, 8 in 2024-09-17T11:55:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:49 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:55:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:49 smithi129 ceph-mon[33364]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:49 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:55:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-132", "mode": "writeback"}]': finished 2024-09-17T11:55:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:49 smithi129 ceph-mon[33364]: osdmap e648: 8 total, 8 up, 8 in 2024-09-17T11:55:51.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:50 smithi049 ceph-mon[30017]: osdmap e649: 8 total, 8 up, 8 in 2024-09-17T11:55:51.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:50 smithi049 ceph-mon[40223]: osdmap e649: 8 total, 8 up, 8 in 2024-09-17T11:55:51.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:50 smithi129 ceph-mon[33364]: osdmap e649: 8 total, 8 up, 8 in 2024-09-17T11:55:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[30017]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[30017]: osdmap e650: 8 total, 8 up, 8 in 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[40223]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[40223]: osdmap e650: 8 total, 8 up, 8 in 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:51 smithi129 ceph-mon[33364]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:55:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:51 smithi129 ceph-mon[33364]: osdmap e650: 8 total, 8 up, 8 in 2024-09-17T11:55:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[40223]: osdmap e651: 8 total, 8 up, 8 in 2024-09-17T11:55:53.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[30017]: osdmap e651: 8 total, 8 up, 8 in 2024-09-17T11:55:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:52 smithi129 ceph-mon[33364]: osdmap e651: 8 total, 8 up, 8 in 2024-09-17T11:55:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: osdmap e652: 8 total, 8 up, 8 in 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:55:54.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:53 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:54.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:54.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: osdmap e652: 8 total, 8 up, 8 in 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: osdmap e652: 8 total, 8 up, 8 in 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:55:54.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:55:54.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:53 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:54 smithi129 ceph-mon[33364]: osdmap e653: 8 total, 8 up, 8 in 2024-09-17T11:55:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:55.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[40223]: osdmap e653: 8 total, 8 up, 8 in 2024-09-17T11:55:55.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:55.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:55.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[30017]: osdmap e653: 8 total, 8 up, 8 in 2024-09-17T11:55:55.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:55.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:55:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:55 smithi129 ceph-mon[33364]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:55 smithi129 ceph-mon[33364]: osdmap e654: 8 total, 8 up, 8 in 2024-09-17T11:55:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[40223]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:56.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[40223]: osdmap e654: 8 total, 8 up, 8 in 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[30017]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[30017]: osdmap e654: 8 total, 8 up, 8 in 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"} : dispatch 2024-09-17T11:55:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:55:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:55:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:56 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:56 smithi129 ceph-mon[33364]: osdmap e655: 8 total, 8 up, 8 in 2024-09-17T11:55:57.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:57.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:57.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[30017]: osdmap e655: 8 total, 8 up, 8 in 2024-09-17T11:55:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:55:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-132"}]': finished 2024-09-17T11:55:57.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:56 smithi049 ceph-mon[40223]: osdmap e655: 8 total, 8 up, 8 in 2024-09-17T11:55:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:57 smithi129 ceph-mon[33364]: pgmap v1000: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:57 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:57 smithi129 ceph-mon[33364]: osdmap e656: 8 total, 8 up, 8 in 2024-09-17T11:55:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:57 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[40223]: pgmap v1000: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:58.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[40223]: osdmap e656: 8 total, 8 up, 8 in 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[30017]: pgmap v1000: 204 pgs: 204 active+clean; 583 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[30017]: osdmap e656: 8 total, 8 up, 8 in 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:55:58.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:55:58.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:55:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:55:59.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[40223]: osdmap e657: 8 total, 8 up, 8 in 2024-09-17T11:55:59.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:59.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[30017]: osdmap e657: 8 total, 8 up, 8 in 2024-09-17T11:55:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:59.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:59.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:58 smithi129 ceph-mon[33364]: osdmap e657: 8 total, 8 up, 8 in 2024-09-17T11:55:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:55:59.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:00.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: osdmap e658: 8 total, 8 up, 8 in 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: osdmap e659: 8 total, 8 up, 8 in 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:00.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: osdmap e658: 8 total, 8 up, 8 in 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: osdmap e659: 8 total, 8 up, 8 in 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:00.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:55:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:00.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: osdmap e658: 8 total, 8 up, 8 in 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: osdmap e659: 8 total, 8 up, 8 in 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:00.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:55:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:01 smithi129 ceph-mon[33364]: osdmap e660: 8 total, 8 up, 8 in 2024-09-17T11:56:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:01 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:01 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:01 smithi129 ceph-mon[33364]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:56:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:01.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[40223]: osdmap e660: 8 total, 8 up, 8 in 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[40223]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[30017]: osdmap e660: 8 total, 8 up, 8 in 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"} : dispatch 2024-09-17T11:56:01.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:01 smithi049 ceph-mon[30017]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:56:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:02.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"}]': finished 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[30017]: osdmap e661: 8 total, 8 up, 8 in 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"}]': finished 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[40223]: osdmap e661: 8 total, 8 up, 8 in 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:02.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:02 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:02 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:02.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-134", "mode": "writeback"}]': finished 2024-09-17T11:56:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:02 smithi129 ceph-mon[33364]: osdmap e661: 8 total, 8 up, 8 in 2024-09-17T11:56:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:02 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:02.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:02 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[40223]: osdmap e662: 8 total, 8 up, 8 in 2024-09-17T11:56:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[40223]: pgmap v1010: 204 pgs: 204 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[30017]: osdmap e662: 8 total, 8 up, 8 in 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:03 smithi049 ceph-mon[30017]: pgmap v1010: 204 pgs: 204 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-17T11:56:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:03 smithi129 ceph-mon[33364]: osdmap e662: 8 total, 8 up, 8 in 2024-09-17T11:56:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:03 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:03 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"} : dispatch 2024-09-17T11:56:03.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:03 smithi129 ceph-mon[33364]: pgmap v1010: 204 pgs: 204 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-09-17T11:56:04.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:04.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:04.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[30017]: osdmap e663: 8 total, 8 up, 8 in 2024-09-17T11:56:04.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[40223]: osdmap e663: 8 total, 8 up, 8 in 2024-09-17T11:56:04.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:04 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:04.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:04 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:04 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-134"}]': finished 2024-09-17T11:56:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:04 smithi129 ceph-mon[33364]: osdmap e663: 8 total, 8 up, 8 in 2024-09-17T11:56:04.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:04 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:05 smithi049 ceph-mon[30017]: osdmap e664: 8 total, 8 up, 8 in 2024-09-17T11:56:05.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:05 smithi049 ceph-mon[30017]: pgmap v1013: 172 pgs: 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:56:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:05 smithi049 ceph-mon[40223]: osdmap e664: 8 total, 8 up, 8 in 2024-09-17T11:56:05.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:05 smithi049 ceph-mon[40223]: pgmap v1013: 172 pgs: 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:56:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:05 smithi129 ceph-mon[33364]: osdmap e664: 8 total, 8 up, 8 in 2024-09-17T11:56:05.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:05 smithi129 ceph-mon[33364]: pgmap v1013: 172 pgs: 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-17T11:56:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[40223]: osdmap e665: 8 total, 8 up, 8 in 2024-09-17T11:56:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:06.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[30017]: osdmap e665: 8 total, 8 up, 8 in 2024-09-17T11:56:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:06.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:06 smithi129 ceph-mon[33364]: osdmap e665: 8 total, 8 up, 8 in 2024-09-17T11:56:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:06.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:07.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[40223]: osdmap e666: 8 total, 8 up, 8 in 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[40223]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[30017]: osdmap e666: 8 total, 8 up, 8 in 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:07 smithi049 ceph-mon[30017]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:07.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:07.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:07 smithi129 ceph-mon[33364]: osdmap e666: 8 total, 8 up, 8 in 2024-09-17T11:56:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:07.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:07 smithi129 ceph-mon[33364]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 683 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:08.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:08 smithi129 ceph-mon[33364]: osdmap e667: 8 total, 8 up, 8 in 2024-09-17T11:56:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:08.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[40223]: osdmap e667: 8 total, 8 up, 8 in 2024-09-17T11:56:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:08.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[30017]: osdmap e667: 8 total, 8 up, 8 in 2024-09-17T11:56:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:08.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:09.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: osdmap e668: 8 total, 8 up, 8 in 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"}]': finished 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:09.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: osdmap e668: 8 total, 8 up, 8 in 2024-09-17T11:56:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:09.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"}]': finished 2024-09-17T11:56:09.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: osdmap e668: 8 total, 8 up, 8 in 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"} : dispatch 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: pgmap v1019: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:09.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-136", "mode": "writeback"}]': finished 2024-09-17T11:56:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[40223]: osdmap e669: 8 total, 8 up, 8 in 2024-09-17T11:56:10.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:10.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[30017]: osdmap e669: 8 total, 8 up, 8 in 2024-09-17T11:56:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:10.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:10 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:10 smithi129 ceph-mon[33364]: osdmap e669: 8 total, 8 up, 8 in 2024-09-17T11:56:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:10 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:10.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:10 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[30017]: osdmap e670: 8 total, 8 up, 8 in 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[30017]: pgmap v1022: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[40223]: osdmap e670: 8 total, 8 up, 8 in 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:11 smithi049 ceph-mon[40223]: pgmap v1022: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:11 smithi129 ceph-mon[33364]: osdmap e670: 8 total, 8 up, 8 in 2024-09-17T11:56:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"} : dispatch 2024-09-17T11:56:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:11 smithi129 ceph-mon[33364]: pgmap v1022: 204 pgs: 204 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[40223]: osdmap e671: 8 total, 8 up, 8 in 2024-09-17T11:56:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:12 smithi049 ceph-mon[30017]: osdmap e671: 8 total, 8 up, 8 in 2024-09-17T11:56:12.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:12 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-136"}]': finished 2024-09-17T11:56:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:12 smithi129 ceph-mon[33364]: osdmap e671: 8 total, 8 up, 8 in 2024-09-17T11:56:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[40223]: osdmap e672: 8 total, 8 up, 8 in 2024-09-17T11:56:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[40223]: pgmap v1025: 172 pgs: 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[30017]: osdmap e672: 8 total, 8 up, 8 in 2024-09-17T11:56:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:13 smithi049 ceph-mon[30017]: pgmap v1025: 172 pgs: 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:13 smithi129 ceph-mon[33364]: osdmap e672: 8 total, 8 up, 8 in 2024-09-17T11:56:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:13 smithi129 ceph-mon[33364]: pgmap v1025: 172 pgs: 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[40223]: osdmap e673: 8 total, 8 up, 8 in 2024-09-17T11:56:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[30017]: osdmap e673: 8 total, 8 up, 8 in 2024-09-17T11:56:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:14.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:14 smithi129 ceph-mon[33364]: osdmap e673: 8 total, 8 up, 8 in 2024-09-17T11:56:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:14.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[40223]: osdmap e674: 8 total, 8 up, 8 in 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[40223]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[30017]: osdmap e674: 8 total, 8 up, 8 in 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:15 smithi049 ceph-mon[30017]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:15 smithi129 ceph-mon[33364]: osdmap e674: 8 total, 8 up, 8 in 2024-09-17T11:56:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:15 smithi129 ceph-mon[33364]: pgmap v1028: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-17T11:56:16.802 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:16.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:16.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[30017]: osdmap e675: 8 total, 8 up, 8 in 2024-09-17T11:56:16.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:16.803 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:16.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:16.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[40223]: osdmap e675: 8 total, 8 up, 8 in 2024-09-17T11:56:16.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:16.803 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:16.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:16.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:16 smithi129 ceph-mon[33364]: osdmap e675: 8 total, 8 up, 8 in 2024-09-17T11:56:16.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:16.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:17.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[30017]: osdmap e676: 8 total, 8 up, 8 in 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[30017]: pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[40223]: osdmap e676: 8 total, 8 up, 8 in 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:17 smithi049 ceph-mon[40223]: pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:17.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:17 smithi129 ceph-mon[33364]: osdmap e676: 8 total, 8 up, 8 in 2024-09-17T11:56:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:17.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:17 smithi129 ceph-mon[33364]: pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:18.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:18.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[40223]: osdmap e677: 8 total, 8 up, 8 in 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[30017]: osdmap e677: 8 total, 8 up, 8 in 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:18.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:18.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:18 smithi129 ceph-mon[33364]: osdmap e677: 8 total, 8 up, 8 in 2024-09-17T11:56:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:18.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-17T11:56:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:19.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[40223]: osdmap e678: 8 total, 8 up, 8 in 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[40223]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[30017]: osdmap e678: 8 total, 8 up, 8 in 2024-09-17T11:56:19.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:19 smithi049 ceph-mon[30017]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-17T11:56:19.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:19 smithi129 ceph-mon[33364]: osdmap e678: 8 total, 8 up, 8 in 2024-09-17T11:56:19.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:19 smithi129 ceph-mon[33364]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:20.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:20.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:20.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:20.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"} : dispatch 2024-09-17T11:56:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"}]': finished 2024-09-17T11:56:21.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[40223]: osdmap e679: 8 total, 8 up, 8 in 2024-09-17T11:56:21.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[40223]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"}]': finished 2024-09-17T11:56:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[30017]: osdmap e679: 8 total, 8 up, 8 in 2024-09-17T11:56:21.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:21 smithi049 ceph-mon[30017]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:21.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-138"}]': finished 2024-09-17T11:56:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:21 smithi129 ceph-mon[33364]: osdmap e679: 8 total, 8 up, 8 in 2024-09-17T11:56:21.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:21 smithi129 ceph-mon[33364]: pgmap v1036: 204 pgs: 204 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:22 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:22.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:22 smithi049 ceph-mon[40223]: osdmap e680: 8 total, 8 up, 8 in 2024-09-17T11:56:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:22 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:22.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:22 smithi049 ceph-mon[30017]: osdmap e680: 8 total, 8 up, 8 in 2024-09-17T11:56:22.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:22 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:22.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:22 smithi129 ceph-mon[33364]: osdmap e680: 8 total, 8 up, 8 in 2024-09-17T11:56:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: osdmap e681: 8 total, 8 up, 8 in 2024-09-17T11:56:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[40223]: osdmap e682: 8 total, 8 up, 8 in 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: osdmap e681: 8 total, 8 up, 8 in 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:23.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:23 smithi049 ceph-mon[30017]: osdmap e682: 8 total, 8 up, 8 in 2024-09-17T11:56:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: osdmap e681: 8 total, 8 up, 8 in 2024-09-17T11:56:23.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:23.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:23 smithi129 ceph-mon[33364]: osdmap e682: 8 total, 8 up, 8 in 2024-09-17T11:56:24.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: osdmap e683: 8 total, 8 up, 8 in 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: osdmap e683: 8 total, 8 up, 8 in 2024-09-17T11:56:24.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:24.902 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:24 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:24.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: osdmap e683: 8 total, 8 up, 8 in 2024-09-17T11:56:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:24.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:24 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-17T11:56:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[40223]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:56:25.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[40223]: osdmap e684: 8 total, 8 up, 8 in 2024-09-17T11:56:25.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[30017]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:56:25.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:25 smithi049 ceph-mon[30017]: osdmap e684: 8 total, 8 up, 8 in 2024-09-17T11:56:25.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:25 smithi129 ceph-mon[33364]: pgmap v1042: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:25.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_count","val": "3"}]': finished 2024-09-17T11:56:25.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:25 smithi129 ceph-mon[33364]: osdmap e684: 8 total, 8 up, 8 in 2024-09-17T11:56:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:26.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[40223]: osdmap e685: 8 total, 8 up, 8 in 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[30017]: osdmap e685: 8 total, 8 up, 8 in 2024-09-17T11:56:26.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:26 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:26.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-17T11:56:26.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:26 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_period","val": "3"}]': finished 2024-09-17T11:56:26.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:26 smithi129 ceph-mon[33364]: osdmap e685: 8 total, 8 up, 8 in 2024-09-17T11:56:26.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:26 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[30017]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:27.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:27.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[30017]: osdmap e686: 8 total, 8 up, 8 in 2024-09-17T11:56:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[40223]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:27.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:27 smithi049 ceph-mon[40223]: osdmap e686: 8 total, 8 up, 8 in 2024-09-17T11:56:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:27 smithi129 ceph-mon[33364]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:27.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:27 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:27.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:27 smithi129 ceph-mon[33364]: osdmap e686: 8 total, 8 up, 8 in 2024-09-17T11:56:28.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[30017]: osdmap e687: 8 total, 8 up, 8 in 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:56:28.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:28 smithi049 ceph-mon[40223]: osdmap e687: 8 total, 8 up, 8 in 2024-09-17T11:56:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-17T11:56:28.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-17T11:56:28.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:28 smithi129 ceph-mon[33364]: osdmap e687: 8 total, 8 up, 8 in 2024-09-17T11:56:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[40223]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:29.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:29.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:29.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[30017]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:29.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:29 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:29 smithi129 ceph-mon[33364]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:29.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:29 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:56:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:31 smithi049 ceph-mon[40223]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-09-17T11:56:32.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:31 smithi049 ceph-mon[30017]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-09-17T11:56:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:31 smithi129 ceph-mon[33364]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-09-17T11:56:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:33 smithi049 ceph-mon[40223]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-09-17T11:56:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:33 smithi049 ceph-mon[30017]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-09-17T11:56:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:33 smithi129 ceph-mon[33364]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-09-17T11:56:36.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:35 smithi049 ceph-mon[30017]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:56:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:35 smithi049 ceph-mon[40223]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:56:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:35 smithi129 ceph-mon[33364]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:56:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:37 smithi049 ceph-mon[40223]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 448 B/s rd, 3.6 KiB/s wr, 0 op/s 2024-09-17T11:56:38.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:37 smithi049 ceph-mon[30017]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 448 B/s rd, 3.6 KiB/s wr, 0 op/s 2024-09-17T11:56:38.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:37 smithi129 ceph-mon[33364]: pgmap v1052: 204 pgs: 204 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 448 B/s rd, 3.6 KiB/s wr, 0 op/s 2024-09-17T11:56:38.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:39 smithi049 ceph-mon[40223]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 910 B/s rd, 7.3 KiB/s wr, 1 op/s 2024-09-17T11:56:40.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:39 smithi049 ceph-mon[30017]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 910 B/s rd, 7.3 KiB/s wr, 1 op/s 2024-09-17T11:56:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:39 smithi129 ceph-mon[33364]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 910 B/s rd, 7.3 KiB/s wr, 1 op/s 2024-09-17T11:56:41.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:41.154 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:41.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"} : dispatch 2024-09-17T11:56:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[40223]: pgmap v1054: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-09-17T11:56:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"}]': finished 2024-09-17T11:56:42.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[40223]: osdmap e688: 8 total, 8 up, 8 in 2024-09-17T11:56:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[30017]: pgmap v1054: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-09-17T11:56:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"}]': finished 2024-09-17T11:56:42.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:41 smithi049 ceph-mon[30017]: osdmap e688: 8 total, 8 up, 8 in 2024-09-17T11:56:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:41 smithi129 ceph-mon[33364]: pgmap v1054: 204 pgs: 204 active+clean; 4.5 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-09-17T11:56:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-140"}]': finished 2024-09-17T11:56:42.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:41 smithi129 ceph-mon[33364]: osdmap e688: 8 total, 8 up, 8 in 2024-09-17T11:56:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:42 smithi049 ceph-mon[40223]: osdmap e689: 8 total, 8 up, 8 in 2024-09-17T11:56:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:42 smithi049 ceph-mon[30017]: osdmap e689: 8 total, 8 up, 8 in 2024-09-17T11:56:43.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:42 smithi129 ceph-mon[33364]: osdmap e689: 8 total, 8 up, 8 in 2024-09-17T11:56:43.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[40223]: pgmap v1057: 172 pgs: 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[40223]: osdmap e690: 8 total, 8 up, 8 in 2024-09-17T11:56:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:44.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[30017]: pgmap v1057: 172 pgs: 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[30017]: osdmap e690: 8 total, 8 up, 8 in 2024-09-17T11:56:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:43 smithi129 ceph-mon[33364]: pgmap v1057: 172 pgs: 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:43 smithi129 ceph-mon[33364]: osdmap e690: 8 total, 8 up, 8 in 2024-09-17T11:56:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[40223]: osdmap e691: 8 total, 8 up, 8 in 2024-09-17T11:56:45.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:45.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[30017]: osdmap e691: 8 total, 8 up, 8 in 2024-09-17T11:56:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:45.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:44 smithi129 ceph-mon[33364]: osdmap e691: 8 total, 8 up, 8 in 2024-09-17T11:56:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:45.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[40223]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[40223]: osdmap e692: 8 total, 8 up, 8 in 2024-09-17T11:56:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[30017]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[30017]: osdmap e692: 8 total, 8 up, 8 in 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:45 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:45 smithi129 ceph-mon[33364]: pgmap v1060: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:46.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:45 smithi129 ceph-mon[33364]: osdmap e692: 8 total, 8 up, 8 in 2024-09-17T11:56:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:45 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:45 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[30017]: osdmap e693: 8 total, 8 up, 8 in 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[40223]: osdmap e693: 8 total, 8 up, 8 in 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:46.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:47.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:46 smithi129 ceph-mon[33364]: osdmap e693: 8 total, 8 up, 8 in 2024-09-17T11:56:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:47.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"} : dispatch 2024-09-17T11:56:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"}]': finished 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: osdmap e694: 8 total, 8 up, 8 in 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"}]': finished 2024-09-17T11:56:48.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: osdmap e694: 8 total, 8 up, 8 in 2024-09-17T11:56:48.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: pgmap v1063: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:56:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-142", "mode": "writeback"}]': finished 2024-09-17T11:56:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: osdmap e694: 8 total, 8 up, 8 in 2024-09-17T11:56:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:56:48.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:49.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: osdmap e695: 8 total, 8 up, 8 in 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[40223]: osdmap e696: 8 total, 8 up, 8 in 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: osdmap e695: 8 total, 8 up, 8 in 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:49.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:48 smithi049 ceph-mon[30017]: osdmap e696: 8 total, 8 up, 8 in 2024-09-17T11:56:49.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:56:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: osdmap e695: 8 total, 8 up, 8 in 2024-09-17T11:56:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:56:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:56:49.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:48 smithi129 ceph-mon[33364]: osdmap e696: 8 total, 8 up, 8 in 2024-09-17T11:56:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: osdmap e697: 8 total, 8 up, 8 in 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: osdmap e697: 8 total, 8 up, 8 in 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:56:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: osdmap e697: 8 total, 8 up, 8 in 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:56:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[30017]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[30017]: osdmap e698: 8 total, 8 up, 8 in 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[40223]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[40223]: osdmap e698: 8 total, 8 up, 8 in 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:52.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:51 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:51 smithi129 ceph-mon[33364]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-17T11:56:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:56:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:51 smithi129 ceph-mon[33364]: osdmap e698: 8 total, 8 up, 8 in 2024-09-17T11:56:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:52.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:51 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-17T11:56:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:56:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: osdmap e699: 8 total, 8 up, 8 in 2024-09-17T11:56:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[30017]: osdmap e700: 8 total, 8 up, 8 in 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: osdmap e699: 8 total, 8 up, 8 in 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:56:53.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:52 smithi049 ceph-mon[40223]: osdmap e700: 8 total, 8 up, 8 in 2024-09-17T11:56:53.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-17T11:56:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: osdmap e699: 8 total, 8 up, 8 in 2024-09-17T11:56:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-17T11:56:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-17T11:56:53.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:52 smithi129 ceph-mon[33364]: osdmap e700: 8 total, 8 up, 8 in 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[30017]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[40223]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:54.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:53 smithi129 ceph-mon[33364]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:55.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:55.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: osdmap e701: 8 total, 8 up, 8 in 2024-09-17T11:56:55.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: osdmap e701: 8 total, 8 up, 8 in 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:56:55.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:56:55.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:54 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:56:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:56:55.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: osdmap e701: 8 total, 8 up, 8 in 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:56:55.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:54 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:56:56.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: osdmap e702: 8 total, 8 up, 8 in 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: osdmap e702: 8 total, 8 up, 8 in 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"}]': finished 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: osdmap e702: 8 total, 8 up, 8 in 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-142"} : dispatch 2024-09-17T11:56:56.871 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:56:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:56:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:56:57.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:56 smithi049 ceph-mon[30017]: osdmap e703: 8 total, 8 up, 8 in 2024-09-17T11:56:57.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:56 smithi049 ceph-mon[40223]: osdmap e703: 8 total, 8 up, 8 in 2024-09-17T11:56:57.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:56 smithi129 ceph-mon[33364]: osdmap e703: 8 total, 8 up, 8 in 2024-09-17T11:56:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[40223]: pgmap v1078: 172 pgs: 172 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[40223]: osdmap e704: 8 total, 8 up, 8 in 2024-09-17T11:56:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[30017]: pgmap v1078: 172 pgs: 172 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[30017]: osdmap e704: 8 total, 8 up, 8 in 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:57 smithi129 ceph-mon[33364]: pgmap v1078: 172 pgs: 172 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:56:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:57 smithi129 ceph-mon[33364]: osdmap e704: 8 total, 8 up, 8 in 2024-09-17T11:56:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:56:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:56:58.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:56:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: osdmap e705: 8 total, 8 up, 8 in 2024-09-17T11:56:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: osdmap e706: 8 total, 8 up, 8 in 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: osdmap e705: 8 total, 8 up, 8 in 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: osdmap e706: 8 total, 8 up, 8 in 2024-09-17T11:56:59.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:56:59.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:56:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: osdmap e705: 8 total, 8 up, 8 in 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: osdmap e706: 8 total, 8 up, 8 in 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:56:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: pgmap v1081: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: osdmap e707: 8 total, 8 up, 8 in 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: pgmap v1081: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: osdmap e707: 8 total, 8 up, 8 in 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:56:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: pgmap v1081: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: osdmap e707: 8 total, 8 up, 8 in 2024-09-17T11:57:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:56:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"} : dispatch 2024-09-17T11:57:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"}]': finished 2024-09-17T11:57:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[40223]: osdmap e708: 8 total, 8 up, 8 in 2024-09-17T11:57:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"}]': finished 2024-09-17T11:57:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:00 smithi049 ceph-mon[30017]: osdmap e708: 8 total, 8 up, 8 in 2024-09-17T11:57:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:00 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:00 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-144", "mode": "readproxy"}]': finished 2024-09-17T11:57:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:00 smithi129 ceph-mon[33364]: osdmap e708: 8 total, 8 up, 8 in 2024-09-17T11:57:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:01 smithi049 ceph-mon[40223]: pgmap v1084: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:02.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:01 smithi049 ceph-mon[30017]: pgmap v1084: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:02.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:01 smithi129 ceph-mon[33364]: pgmap v1084: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-17T11:57:04.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:03 smithi129 ceph-mon[33364]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-09-17T11:57:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:03 smithi049 ceph-mon[40223]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-09-17T11:57:04.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:03 smithi049 ceph-mon[30017]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-09-17T11:57:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:05 smithi129 ceph-mon[33364]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-17T11:57:06.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:05 smithi049 ceph-mon[30017]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-17T11:57:06.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:05 smithi049 ceph-mon[40223]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-17T11:57:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:07 smithi129 ceph-mon[33364]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 130 B/s wr, 0 op/s 2024-09-17T11:57:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:07 smithi049 ceph-mon[40223]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 130 B/s wr, 0 op/s 2024-09-17T11:57:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:07 smithi049 ceph-mon[30017]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 783 B/s rd, 130 B/s wr, 0 op/s 2024-09-17T11:57:08.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:09 smithi129 ceph-mon[33364]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-09-17T11:57:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:09 smithi049 ceph-mon[40223]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-09-17T11:57:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:09 smithi049 ceph-mon[30017]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-09-17T11:57:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:11 smithi129 ceph-mon[33364]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:57:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:12.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[30017]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:57:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:12.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:12.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[40223]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-17T11:57:12.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:12.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: osdmap e709: 8 total, 8 up, 8 in 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:13.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: osdmap e709: 8 total, 8 up, 8 in 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: osdmap e709: 8 total, 8 up, 8 in 2024-09-17T11:57:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:13.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: osdmap e710: 8 total, 8 up, 8 in 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:57:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: osdmap e710: 8 total, 8 up, 8 in 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"}]': finished 2024-09-17T11:57:14.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: osdmap e710: 8 total, 8 up, 8 in 2024-09-17T11:57:14.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:14.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:14.402 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-144"} : dispatch 2024-09-17T11:57:15.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:14 smithi049 ceph-mon[40223]: osdmap e711: 8 total, 8 up, 8 in 2024-09-17T11:57:15.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:14 smithi049 ceph-mon[30017]: osdmap e711: 8 total, 8 up, 8 in 2024-09-17T11:57:15.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:14 smithi129 ceph-mon[33364]: osdmap e711: 8 total, 8 up, 8 in 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[30017]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[30017]: osdmap e712: 8 total, 8 up, 8 in 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[40223]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:57:16.302 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[40223]: osdmap e712: 8 total, 8 up, 8 in 2024-09-17T11:57:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.303 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:15 smithi129 ceph-mon[33364]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:57:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:15 smithi129 ceph-mon[33364]: osdmap e712: 8 total, 8 up, 8 in 2024-09-17T11:57:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:17.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[40223]: osdmap e713: 8 total, 8 up, 8 in 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[30017]: osdmap e713: 8 total, 8 up, 8 in 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:17.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:16 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:16 smithi129 ceph-mon[33364]: osdmap e713: 8 total, 8 up, 8 in 2024-09-17T11:57:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:16 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:18.280 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:17 smithi129 ceph-mon[33364]: pgmap v1098: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:57:18.280 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:18.280 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:17 smithi129 ceph-mon[33364]: osdmap e714: 8 total, 8 up, 8 in 2024-09-17T11:57:18.280 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.280 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:17 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[40223]: pgmap v1098: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:57:18.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[40223]: osdmap e714: 8 total, 8 up, 8 in 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[30017]: pgmap v1098: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[30017]: osdmap e714: 8 total, 8 up, 8 in 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:17 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:18.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:19.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:19.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: osdmap e715: 8 total, 8 up, 8 in 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"}]': finished 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[30017]: osdmap e716: 8 total, 8 up, 8 in 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: osdmap e715: 8 total, 8 up, 8 in 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:19.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"}]': finished 2024-09-17T11:57:19.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:18 smithi049 ceph-mon[40223]: osdmap e716: 8 total, 8 up, 8 in 2024-09-17T11:57:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi049-76013-111", "overlaypool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: osdmap e715: 8 total, 8 up, 8 in 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"} : dispatch 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi049-76013-146", "mode": "writeback"}]': finished 2024-09-17T11:57:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:18 smithi129 ceph-mon[33364]: osdmap e716: 8 total, 8 up, 8 in 2024-09-17T11:57:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[40223]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:20.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[30017]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:19 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:19 smithi129 ceph-mon[33364]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:19 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-17T11:57:21.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:57:21.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[40223]: osdmap e717: 8 total, 8 up, 8 in 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[30017]: osdmap e717: 8 total, 8 up, 8 in 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:21.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:20 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:21.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_count","val": "2"}]': finished 2024-09-17T11:57:21.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:20 smithi129 ceph-mon[33364]: osdmap e717: 8 total, 8 up, 8 in 2024-09-17T11:57:21.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:20 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:21.429 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:20 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-17T11:57:22.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[30017]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[30017]: osdmap e718: 8 total, 8 up, 8 in 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[40223]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[40223]: osdmap e718: 8 total, 8 up, 8 in 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:22.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:21 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:21 smithi129 ceph-mon[33364]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T11:57:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_period","val": "600"}]': finished 2024-09-17T11:57:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:21 smithi129 ceph-mon[33364]: osdmap e718: 8 total, 8 up, 8 in 2024-09-17T11:57:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:21 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:22.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:21 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-17T11:57:23.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:23.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:57:23.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[40223]: osdmap e719: 8 total, 8 up, 8 in 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[30017]: osdmap e719: 8 total, 8 up, 8 in 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:23.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:22 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:23.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:22 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-17T11:57:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-17T11:57:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:22 smithi129 ceph-mon[33364]: osdmap e719: 8 total, 8 up, 8 in 2024-09-17T11:57:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:22 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:23.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:22 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-17T11:57:24.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[30017]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:24.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:57:24.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[30017]: osdmap e720: 8 total, 8 up, 8 in 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[40223]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[40223]: osdmap e720: 8 total, 8 up, 8 in 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:24.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:23 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:24.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:23 smithi129 ceph-mon[33364]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-17T11:57:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:23 smithi129 ceph-mon[33364]: osdmap e720: 8 total, 8 up, 8 in 2024-09-17T11:57:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:23 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:23 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-17T11:57:25.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:25 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:57:25.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:25 smithi049 ceph-mon[40223]: osdmap e721: 8 total, 8 up, 8 in 2024-09-17T11:57:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:25 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:57:25.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:25 smithi049 ceph-mon[30017]: osdmap e721: 8 total, 8 up, 8 in 2024-09-17T11:57:25.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:25 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-146","var": "target_max_objects","val": "1"}]': finished 2024-09-17T11:57:25.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:25 smithi129 ceph-mon[33364]: osdmap e721: 8 total, 8 up, 8 in 2024-09-17T11:57:26.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:26 smithi049 ceph-mon[30017]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:26.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:26 smithi049 ceph-mon[30017]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:57:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:26 smithi049 ceph-mon[40223]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:26.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:26 smithi049 ceph-mon[40223]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:57:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:26 smithi129 ceph-mon[33364]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:26 smithi129 ceph-mon[33364]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-17T11:57:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:28.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:28 smithi129 ceph-mon[33364]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:57:28.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:28 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:28.290 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:28 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:28.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[40223]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:57:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[30017]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-17T11:57:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:28.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:28 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:28.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:30 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:30 smithi049 ceph-mon[40223]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:30 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:30 smithi049 ceph-mon[30017]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:30.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:30 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:30 smithi129 ceph-mon[33364]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-17T11:57:32.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:32 smithi049 ceph-mon[30017]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-17T11:57:32.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:32 smithi049 ceph-mon[40223]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-17T11:57:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:32 smithi129 ceph-mon[33364]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-17T11:57:34.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:34 smithi049 ceph-mon[30017]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:34.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:34 smithi049 ceph-mon[40223]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:34 smithi129 ceph-mon[33364]: pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:35.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: osdmap e722: 8 total, 8 up, 8 in 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:35.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: osdmap e722: 8 total, 8 up, 8 in 2024-09-17T11:57:35.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:35.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:35 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:35.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: osdmap e722: 8 total, 8 up, 8 in 2024-09-17T11:57:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:35.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:35 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: osdmap e723: 8 total, 8 up, 8 in 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: osdmap e723: 8 total, 8 up, 8 in 2024-09-17T11:57:36.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"}]': finished 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: osdmap e723: 8 total, 8 up, 8 in 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:36 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-146"} : dispatch 2024-09-17T11:57:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:37.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:37 smithi049 ceph-mon[40223]: osdmap e724: 8 total, 8 up, 8 in 2024-09-17T11:57:37.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:37 smithi049 ceph-mon[30017]: osdmap e724: 8 total, 8 up, 8 in 2024-09-17T11:57:37.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:37 smithi129 ceph-mon[33364]: osdmap e724: 8 total, 8 up, 8 in 2024-09-17T11:57:38.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[30017]: pgmap v1119: 172 pgs: 172 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:57:38.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[30017]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[30017]: osdmap e725: 8 total, 8 up, 8 in 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[40223]: pgmap v1119: 172 pgs: 172 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[40223]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[40223]: osdmap e725: 8 total, 8 up, 8 in 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:38.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:38 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:38.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-mon[33364]: pgmap v1119: 172 pgs: 172 active+clean; 4.4 MiB data, 771 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-17T11:57:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-mon[33364]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-17T11:57:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-mon[33364]: osdmap e725: 8 total, 8 up, 8 in 2024-09-17T11:57:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:38.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:38 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:39.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[30017]: osdmap e726: 8 total, 8 up, 8 in 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[40223]: osdmap e726: 8 total, 8 up, 8 in 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:39.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:39 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:39.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:39 smithi129 ceph-mon[33364]: osdmap e726: 8 total, 8 up, 8 in 2024-09-17T11:57:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:39 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:39.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:39 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-17T11:57:40.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: pgmap v1122: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: osdmap e727: 8 total, 8 up, 8 in 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: pgmap v1122: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: osdmap e727: 8 total, 8 up, 8 in 2024-09-17T11:57:40.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:40.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:40 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: pgmap v1122: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-17T11:57:40.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-17T11:57:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: osdmap e727: 8 total, 8 up, 8 in 2024-09-17T11:57:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:40.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:40 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"}]': finished 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: osdmap e728: 8 total, 8 up, 8 in 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"}]': finished 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: osdmap e728: 8 total, 8 up, 8 in 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:41 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"}]': finished 2024-09-17T11:57:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: osdmap e728: 8 total, 8 up, 8 in 2024-09-17T11:57:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:41.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:41 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-148"} : dispatch 2024-09-17T11:57:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[40223]: pgmap v1125: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:57:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[40223]: osdmap e729: 8 total, 8 up, 8 in 2024-09-17T11:57:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[30017]: pgmap v1125: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:57:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:42.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:42 smithi049 ceph-mon[30017]: osdmap e729: 8 total, 8 up, 8 in 2024-09-17T11:57:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:42 smithi129 ceph-mon[33364]: pgmap v1125: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:57:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:42 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:42 smithi129 ceph-mon[33364]: osdmap e729: 8 total, 8 up, 8 in 2024-09-17T11:57:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[40223]: osdmap e730: 8 total, 8 up, 8 in 2024-09-17T11:57:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[30017]: osdmap e730: 8 total, 8 up, 8 in 2024-09-17T11:57:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:43.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:43 smithi129 ceph-mon[33364]: osdmap e730: 8 total, 8 up, 8 in 2024-09-17T11:57:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:43 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:43 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:43 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:43.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:44.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: pgmap v1128: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:44.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: osdmap e731: 8 total, 8 up, 8 in 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: pgmap v1128: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: osdmap e731: 8 total, 8 up, 8 in 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:44.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:44 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:44.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: pgmap v1128: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:44.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: osdmap e731: 8 total, 8 up, 8 in 2024-09-17T11:57:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:44.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:44 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-150"} : dispatch 2024-09-17T11:57:45.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:45 smithi049 ceph-mon[30017]: osdmap e732: 8 total, 8 up, 8 in 2024-09-17T11:57:45.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:45 smithi049 ceph-mon[40223]: osdmap e732: 8 total, 8 up, 8 in 2024-09-17T11:57:45.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:45 smithi129 ceph-mon[33364]: osdmap e732: 8 total, 8 up, 8 in 2024-09-17T11:57:46.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:46 smithi129 ceph-mon[33364]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:46 smithi129 ceph-mon[33364]: osdmap e733: 8 total, 8 up, 8 in 2024-09-17T11:57:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:46 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:46 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[30017]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[30017]: osdmap e733: 8 total, 8 up, 8 in 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[40223]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[40223]: osdmap e733: 8 total, 8 up, 8 in 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.611 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:47.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: osdmap e734: 8 total, 8 up, 8 in 2024-09-17T11:57:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:47.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:47 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: osdmap e734: 8 total, 8 up, 8 in 2024-09-17T11:57:47.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: osdmap e734: 8 total, 8 up, 8 in 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:47.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:47 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-152"} : dispatch 2024-09-17T11:57:48.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:48 smithi129 ceph-mon[33364]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:48.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:48 smithi129 ceph-mon[33364]: osdmap e735: 8 total, 8 up, 8 in 2024-09-17T11:57:48.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:48 smithi049 ceph-mon[40223]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:48.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:48 smithi049 ceph-mon[40223]: osdmap e735: 8 total, 8 up, 8 in 2024-09-17T11:57:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:48 smithi049 ceph-mon[30017]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 776 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:48.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:48 smithi049 ceph-mon[30017]: osdmap e735: 8 total, 8 up, 8 in 2024-09-17T11:57:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[30017]: osdmap e736: 8 total, 8 up, 8 in 2024-09-17T11:57:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:49.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[40223]: osdmap e736: 8 total, 8 up, 8 in 2024-09-17T11:57:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:49.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:49 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:49 smithi129 ceph-mon[33364]: osdmap e736: 8 total, 8 up, 8 in 2024-09-17T11:57:49.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:49.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:49 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T11:57:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:50.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: pgmap v1137: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: osdmap e737: 8 total, 8 up, 8 in 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: pgmap v1137: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: osdmap e737: 8 total, 8 up, 8 in 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:50 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:50.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: pgmap v1137: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi049-76013-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: osdmap e737: 8 total, 8 up, 8 in 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi049-76013-111","var": "dedup_tier","val": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:50.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:50 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi049-76013-111", "tierpool": "test-rados-api-smithi049-76013-154"} : dispatch 2024-09-17T11:57:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:51 smithi049 ceph-mon[40223]: osdmap e738: 8 total, 8 up, 8 in 2024-09-17T11:57:51.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:51 smithi049 ceph-mon[40223]: pgmap v1140: 172 pgs: 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:51.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:51 smithi049 ceph-mon[30017]: osdmap e738: 8 total, 8 up, 8 in 2024-09-17T11:57:51.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:51 smithi049 ceph-mon[30017]: pgmap v1140: 172 pgs: 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:51.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:51 smithi129 ceph-mon[33364]: osdmap e738: 8 total, 8 up, 8 in 2024-09-17T11:57:51.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:51 smithi129 ceph-mon[33364]: pgmap v1140: 172 pgs: 172 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-17T11:57:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[30017]: osdmap e739: 8 total, 8 up, 8 in 2024-09-17T11:57:52.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:52.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[40223]: osdmap e739: 8 total, 8 up, 8 in 2024-09-17T11:57:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:52.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:52 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:52.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:52 smithi129 ceph-mon[33364]: osdmap e739: 8 total, 8 up, 8 in 2024-09-17T11:57:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:52 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:52.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:52 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.216 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (8050 ms) 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12297 ms) 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8064 ms) 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13092 ms) 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7518 ms) 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-17T11:57:53.217 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8055 ms) 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9070 ms) 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: first is 1726574189 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,0 2024-09-17T11:57:53.218 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,1726574197,1726574198,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,1726574197,1726574198,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574189,1726574191,1726574192,1726574194,1726574195,1726574197,1726574198,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: got ls 1726574192,1726574194,1726574195,1726574197,1726574198,1726574200,1726574201,0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: first now 1726574192, trimmed 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20218 ms) 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: foo0 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14086 ms) 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18068 ms) 2024-09-17T11:57:53.219 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22142 ms) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5036 ms) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3016 ms) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3031 ms) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3025 ms) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (230152 ms total) 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: 2024-09-17T11:57:53.220 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-17T11:57:53.221 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (845800 ms total) 2024-09-17T11:57:53.221 INFO:tasks.workunit.client.0.smithi049.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-17T11:57:53.226 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.226 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75696 2024-09-17T11:57:53.226 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75696 2024-09-17T11:57:53.226 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.226 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75951 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75951 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76456 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76456 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76205 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76205 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76022 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76022 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.227 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75780 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75780 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76106 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76106 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76579 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76579 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76618 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76618 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75802 2024-09-17T11:57:53.228 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75802 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76472 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76472 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75670 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75670 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75733 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75733 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76408 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76408 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.229 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75661 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75661 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75682 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75682 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76791 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76791 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75980 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75980 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76966 2024-09-17T11:57:53.230 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76966 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76057 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76057 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76572 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76572 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=77022 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 77022 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75995 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75995 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.231 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76425 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76425 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76714 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76714 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76547 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76547 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76888 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76888 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76380 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76380 2024-09-17T11:57:53.232 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75891 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75891 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76993 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76993 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76228 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76228 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75654 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75654 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75714 2024-09-17T11:57:53.233 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75714 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76120 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76120 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=75756 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 75756 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76254 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76254 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76333 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76333 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.234 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=76696 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 76696 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ for t in "${!pids[@]}" 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=77067 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 77067 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ exit 0 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ cleanup 2024-09-17T11:57:53.235 INFO:tasks.workunit.client.0.smithi049.stderr:+ pkill -P 75648 2024-09-17T11:57:53.238 INFO:tasks.workunit.client.0.smithi049.stderr:+ true 2024-09-17T11:57:53.238 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-17T11:57:53.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-17T11:57:53.270 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-17T11:57:53.272 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-17T11:57:53.272 DEBUG:teuthology.orchestra.run.smithi049:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2b698643d05f73e79c2d2888228947a84a4ac0bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-17T11:57:53.332 INFO:tasks.workunit.client.0.smithi049.stderr:++ uuidgen 2024-09-17T11:57:53.340 INFO:tasks.workunit.client.0.smithi049.stderr:+ p=b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:57:53.340 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool create b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 12 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 -- 172.21.15.49:0/3887820188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c06a540 msgr2=0x7ff24c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 --2- 172.21.15.49:0/3887820188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c06a540 0x7ff24c06a920 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7ff248009860 tx=0x7ff24802a340 comp rx=0 tx=0).stop 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 -- 172.21.15.49:0/3887820188 shutdown_connections 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 --2- 172.21.15.49:0/3887820188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c109050 0x7ff24c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 --2- 172.21.15.49:0/3887820188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c1086d0 0x7ff24c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 --2- 172.21.15.49:0/3887820188 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c06a540 0x7ff24c06a920 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 -- 172.21.15.49:0/3887820188 >> 172.21.15.49:0/3887820188 conn(0x7ff24c0776a0 msgr2=0x7ff24c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 -- 172.21.15.49:0/3887820188 shutdown_connections 2024-09-17T11:57:53.429 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.424+0000 7ff252edd640 1 -- 172.21.15.49:0/3887820188 wait complete. 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 Processor -- start 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 -- start start 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c1086d0 0x7ff24c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c109050 0x7ff24c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c112420 con 0x7ff24c06a540 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1122a0 con 0x7ff24c1086d0 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff252edd640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c1125a0 con 0x7ff24c109050 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff250c79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff250c79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47480/0 (socket says 172.21.15.49:47480) 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff250c79640 1 -- 172.21.15.49:0/1027433735 learned_addr learned my addr 172.21.15.49:0/1027433735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:57:53.430 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.425+0000 7ff25147a640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c109050 0x7ff24c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 -- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c109050 msgr2=0x7ff24c19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c109050 0x7ff24c19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff243fff640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c1086d0 0x7ff24c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 -- 172.21.15.49:0/1027433735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c1086d0 msgr2=0x7ff24c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c1086d0 0x7ff24c198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19e9a0 con 0x7ff24c06a540 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff250c79640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7ff248009980 tx=0x7ff248033d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24803f020 con 0x7ff24c06a540 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7ff2480044c0 con 0x7ff24c06a540 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19ec30 con 0x7ff24c06a540 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248037040 con 0x7ff24c06a540 2024-09-17T11:57:53.431 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.426+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a5350 con 0x7ff24c06a540 2024-09-17T11:57:53.432 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.427+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c06a920 con 0x7ff24c06a540 2024-09-17T11:57:53.432 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.428+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7ff248042020 con 0x7ff24c06a540 2024-09-17T11:57:53.432 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.429+0000 7ff241ffb640 1 --2- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 0x7ff23407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:53.433 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.429+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(741..741 src has 1..741) ==== 7370+0+0 (secure 0 0 0) 0x7ff2480be410 con 0x7ff24c06a540 2024-09-17T11:57:53.433 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.429+0000 7ff243fff640 1 --2- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 0x7ff23407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:53.433 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.429+0000 7ff243fff640 1 --2- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 0x7ff23407d290 secure :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0096c0 tx=0x7ff23c009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:53.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.434+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7ff248049020 con 0x7ff24c06a540 2024-09-17T11:57:53.624 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:53.622+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} v 0) -- 0x7ff24c10ae30 con 0x7ff24c06a540 2024-09-17T11:57:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[30017]: osdmap e740: 8 total, 8 up, 8 in 2024-09-17T11:57:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[30017]: pgmap v1143: 164 pgs: 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[40223]: osdmap e740: 8 total, 8 up, 8 in 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:53 smithi049 ceph-mon[40223]: pgmap v1143: 164 pgs: 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:53.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:53 smithi129 ceph-mon[33364]: osdmap e740: 8 total, 8 up, 8 in 2024-09-17T11:57:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:53 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/931280206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:53 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"} : dispatch 2024-09-17T11:57:53.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:53 smithi129 ceph-mon[33364]: pgmap v1143: 164 pgs: 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:54.212 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.210+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12}]=0 pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' created v742) ==== 176+0+0 (secure 0 0 0) 0x7ff2480896f0 con 0x7ff24c06a540 2024-09-17T11:57:54.334 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.332+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} v 0) -- 0x7ff24c10a9c0 con 0x7ff24c06a540 2024-09-17T11:57:54.334 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.332+0000 7ff241ffb640 1 -- 172.21.15.49:0/1027433735 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12}]=0 pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' already exists v742) ==== 183+0+0 (secure 0 0 0) 0x7ff24808e5a0 con 0x7ff24c06a540 2024-09-17T11:57:54.334 INFO:tasks.workunit.client.0.smithi049.stderr:pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' already exists 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 msgr2=0x7ff23407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 0x7ff23407d290 secure :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0096c0 tx=0x7ff23c009260 comp rx=0 tx=0).stop 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 msgr2=0x7ff24c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7ff248009980 tx=0x7ff248033d30 comp rx=0 tx=0).stop 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 shutdown_connections 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> v2:172.21.15.49:6800/1163045599 conn(0x7ff23407add0 0x7ff23407d290 unknown :-1 s=CLOSED pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.337 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7ff24c109050 0x7ff24c19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.338 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff24c1086d0 0x7ff24c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.338 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.334+0000 7ff252edd640 1 --2- 172.21.15.49:0/1027433735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff24c06a540 0x7ff24c198900 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.338 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.335+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 >> 172.21.15.49:0/1027433735 conn(0x7ff24c0776a0 msgr2=0x7ff24c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:54.338 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.335+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 shutdown_connections 2024-09-17T11:57:54.338 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.335+0000 7ff252edd640 1 -- 172.21.15.49:0/1027433735 wait complete. 2024-09-17T11:57:54.346 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_objects 10 2024-09-17T11:57:54.434 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 -- 172.21.15.49:0/3055469048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 msgr2=0x7fe1981116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:54.434 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 --2- 172.21.15.49:0/3055469048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981116e0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fe190009ca0 tx=0x7fe19002a8c0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.434 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 -- 172.21.15.49:0/3055469048 shutdown_connections 2024-09-17T11:57:54.434 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 --2- 172.21.15.49:0/3055469048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981116e0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 --2- 172.21.15.49:0/3055469048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 0x7fe19810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 --2- 172.21.15.49:0/3055469048 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe1980758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 -- 172.21.15.49:0/3055469048 >> 172.21.15.49:0/3055469048 conn(0x7fe1980fe750 msgr2=0x7fe198100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 -- 172.21.15.49:0/3055469048 shutdown_connections 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.429+0000 7fe19e53d640 1 -- 172.21.15.49:0/3055469048 wait complete. 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 Processor -- start 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 -- start start 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 0x7fe19819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:54.435 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981145e0 con 0x7fe19810f2f0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe198114460 con 0x7fe198075de0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19e53d640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fe198114760 con 0x7fe1980754c0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:34338/0 (socket says 172.21.15.49:34338) 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 -- 172.21.15.49:0/915147738 learned_addr learned my addr 172.21.15.49:0/915147738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe19cada640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe18ffff640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 0x7fe19819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 -- 172.21.15.49:0/915147738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 msgr2=0x7fe19819b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 0x7fe19819b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 -- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 msgr2=0x7fe1981a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:54.436 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.430+0000 7fe197fff640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1981a0b80 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe197fff640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fe18800ea60 tx=0x7fe18800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18800cde0 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fe1880035f0 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1981a0e70 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880044a0 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.431+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a7530 con 0x7fe1980754c0 2024-09-17T11:57:54.437 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.432+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1980758a0 con 0x7fe1980754c0 2024-09-17T11:57:54.438 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.433+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fe188004000 con 0x7fe1980754c0 2024-09-17T11:57:54.438 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.433+0000 7fe195ffb640 1 --2- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 0x7fe18407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:54.438 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.433+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(742..742 src has 1..742) ==== 7745+0+0 (secure 0 0 0) 0x7fe18809e9a0 con 0x7fe1980754c0 2024-09-17T11:57:54.438 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.433+0000 7fe18ffff640 1 --2- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 0x7fe18407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:54.438 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.433+0000 7fe18ffff640 1 --2- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 0x7fe18407d290 secure :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0x7fe1981118e0 tx=0x7fe18000a4c0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:54.440 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.438+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fe188069b00 con 0x7fe1980754c0 2024-09-17T11:57:54.629 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:54.627+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} v 0) -- 0x7fe19819bc00 con 0x7fe1980754c0 2024-09-17T11:57:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[40223]: osdmap e741: 8 total, 8 up, 8 in 2024-09-17T11:57:54.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[30017]: osdmap e741: 8 total, 8 up, 8 in 2024-09-17T11:57:54.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:54 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:54 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi049-76013-111"}]': finished 2024-09-17T11:57:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:54 smithi129 ceph-mon[33364]: osdmap e741: 8 total, 8 up, 8 in 2024-09-17T11:57:54.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:54 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:55.222 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:55.219+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v743) ==== 223+0+0 (secure 0 0 0) 0x7fe18806e9b0 con 0x7fe1980754c0 2024-09-17T11:57:55.346 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:55.344+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} v 0) -- 0x7fe198108f00 con 0x7fe1980754c0 2024-09-17T11:57:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12}]': finished 2024-09-17T11:57:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: osdmap e742: 8 total, 8 up, 8 in 2024-09-17T11:57:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:55.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12}]': finished 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: osdmap e742: 8 total, 8 up, 8 in 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:55.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:55 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:57:55.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12}]': finished 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: osdmap e742: 8 total, 8 up, 8 in 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1027433735' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pg_num": 12} : dispatch 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: pgmap v1146: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:55.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:55 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:57:56.225 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.224+0000 7fe195ffb640 1 -- 172.21.15.49:0/915147738 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v744) ==== 223+0+0 (secure 0 0 0) 0x7fe18809c7e0 con 0x7fe1980754c0 2024-09-17T11:57:56.226 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_objects = 10 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 msgr2=0x7fe18407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 0x7fe18407d290 secure :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0x7fe1981118e0 tx=0x7fe18000a4c0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 msgr2=0x7fe19819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fe18800ea60 tx=0x7fe18800ef20 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 shutdown_connections 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> v2:172.21.15.49:6800/1163045599 conn(0x7fe18407add0 0x7fe18407d290 unknown :-1 s=CLOSED pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe19810f2f0 0x7fe1981a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe198075de0 0x7fe19819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 --2- 172.21.15.49:0/915147738 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fe1980754c0 0x7fe19819aae0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 >> 172.21.15.49:0/915147738 conn(0x7fe1980fe750 msgr2=0x7fe1980ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 shutdown_connections 2024-09-17T11:57:56.229 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.227+0000 7fe19e53d640 1 -- 172.21.15.49:0/915147738 wait complete. 2024-09-17T11:57:56.238 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool application enable b9582b3b-e4bf-449f-bb78-cbf577bc7b0a rados 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 -- 172.21.15.49:0/2331230643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 msgr2=0x7f6438101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 --2- 172.21.15.49:0/2331230643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f6438101fd0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f6428009860 tx=0x7f642802a340 comp rx=0 tx=0).stop 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 -- 172.21.15.49:0/2331230643 shutdown_connections 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 --2- 172.21.15.49:0/2331230643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 --2- 172.21.15.49:0/2331230643 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 --2- 172.21.15.49:0/2331230643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f6438101fd0 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 -- 172.21.15.49:0/2331230643 >> 172.21.15.49:0/2331230643 conn(0x7f64380fd630 msgr2=0x7f64380ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 -- 172.21.15.49:0/2331230643 shutdown_connections 2024-09-17T11:57:56.326 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.322+0000 7f643ca60640 1 -- 172.21.15.49:0/2331230643 wait complete. 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 Processor -- start 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 -- start start 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111300 con 0x7f6438101bf0 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111180 con 0x7f6438107f70 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643ca60640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f6438111480 con 0x7f64381075f0 2024-09-17T11:57:56.327 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47506/0 (socket says 172.21.15.49:47506) 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:34350/0 (socket says 172.21.15.49:34350) 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643659c640 1 -- 172.21.15.49:0/3395853583 learned_addr learned my addr 172.21.15.49:0/3395853583 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f6436d9d640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 -- 172.21.15.49:0/3395853583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 msgr2=0x7f643819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 -- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 msgr2=0x7f64381988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643819e990 con 0x7f64381075f0 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f6436d9d640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:57:56.328 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f643659c640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.323+0000 7f642ffff640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.324+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000cd00 con 0x7f64381075f0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.324+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f64200035f0 con 0x7f64381075f0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.324+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643819ec80 con 0x7f64381075f0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.324+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200044a0 con 0x7f64381075f0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.324+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381a5340 con 0x7f64381075f0 2024-09-17T11:57:56.329 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.325+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438101fd0 con 0x7f64381075f0 2024-09-17T11:57:56.330 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.325+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f6420004000 con 0x7f64381075f0 2024-09-17T11:57:56.330 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.326+0000 7f642f7fe640 1 --2- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 0x7f642407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:57:56.330 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.326+0000 7f643659c640 1 --2- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 0x7f642407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:57:56.330 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.326+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(744..744 src has 1..744) ==== 7745+0+0 (secure 0 0 0) 0x7f642009e780 con 0x7f64381075f0 2024-09-17T11:57:56.330 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.327+0000 7f643659c640 1 --2- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 0x7f642407d290 secure :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0x7f64280095f0 tx=0x7f64280056d0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:57:56.332 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.331+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f64200698e0 con 0x7f64381075f0 2024-09-17T11:57:56.522 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:56.521+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} v 0) -- 0x7f64381a5600 con 0x7f64381075f0 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: osdmap e743: 8 total, 8 up, 8 in 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.612 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: osdmap e743: 8 total, 8 up, 8 in 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.613 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:57:56.614 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:57:56.614 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: osdmap e743: 8 total, 8 up, 8 in 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/915147738' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:57:56.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:57:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:57:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:57:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:57:57.233 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:57.232+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]=0 enabled application 'rados' on pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' v745) ==== 213+0+0 (secure 0 0 0) 0x7f642006e790 con 0x7f64381075f0 2024-09-17T11:57:57.350 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:57.349+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} v 0) -- 0x7f64381a5850 con 0x7f64381075f0 2024-09-17T11:57:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[40223]: osdmap e744: 8 total, 8 up, 8 in 2024-09-17T11:57:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[40223]: pgmap v1149: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[30017]: osdmap e744: 8 total, 8 up, 8 in 2024-09-17T11:57:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:57 smithi049 ceph-mon[30017]: pgmap v1149: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:57.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:57:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:57 smithi129 ceph-mon[33364]: osdmap e744: 8 total, 8 up, 8 in 2024-09-17T11:57:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:57 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:57 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:57.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:57 smithi129 ceph-mon[33364]: pgmap v1149: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-09-17T11:57:58.235 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.233+0000 7f642f7fe640 1 -- 172.21.15.49:0/3395853583 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]=0 enabled application 'rados' on pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' v746) ==== 213+0+0 (secure 0 0 0) 0x7f642009c5c0 con 0x7f64381075f0 2024-09-17T11:57:58.235 INFO:tasks.workunit.client.0.smithi049.stderr:enabled application 'rados' on pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' 2024-09-17T11:57:58.239 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 msgr2=0x7f642407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:58.239 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 0x7f642407d290 secure :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0x7f64280095f0 tx=0x7f64280056d0 comp rx=0 tx=0).stop 2024-09-17T11:57:58.239 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 msgr2=0x7f6438198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000ee50 comp rx=0 tx=0).stop 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 shutdown_connections 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> v2:172.21.15.49:6800/1163045599 conn(0x7f642407add0 0x7f642407d290 unknown :-1 s=CLOSED pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6438107f70 0x7f643819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f64381075f0 0x7f6438198e30 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 --2- 172.21.15.49:0/3395853583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6438101bf0 0x7f64381988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 >> 172.21.15.49:0/3395853583 conn(0x7f64380fd630 msgr2=0x7f643806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 shutdown_connections 2024-09-17T11:57:58.240 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:57:58.238+0000 7f643ca60640 1 -- 172.21.15.49:0/3395853583 wait complete. 2024-09-17T11:57:58.251 INFO:tasks.workunit.client.0.smithi049.stderr:++ seq 1 10 2024-09-17T11:57:58.266 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.266 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj1 /etc/passwd 2024-09-17T11:57:58.312 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.312 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj2 /etc/passwd 2024-09-17T11:57:58.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.356 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj3 /etc/passwd 2024-09-17T11:57:58.395 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.395 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj4 /etc/passwd 2024-09-17T11:57:58.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.432 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj5 /etc/passwd 2024-09-17T11:57:58.469 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.470 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj6 /etc/passwd 2024-09-17T11:57:58.506 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.507 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj7 /etc/passwd 2024-09-17T11:57:58.543 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.544 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj8 /etc/passwd 2024-09-17T11:57:58.580 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.581 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj9 /etc/passwd 2024-09-17T11:57:58.617 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:57:58.617 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put obj10 /etc/passwd 2024-09-17T11:57:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[40223]: osdmap e745: 8 total, 8 up, 8 in 2024-09-17T11:57:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.650 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[30017]: osdmap e745: 8 total, 8 up, 8 in 2024-09-17T11:57:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:58 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:58.654 INFO:tasks.workunit.client.0.smithi049.stderr:+ sleep 30 2024-09-17T11:57:58.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:57:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-mon[33364]: osdmap e745: 8 total, 8 up, 8 in 2024-09-17T11:57:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3395853583' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"} : dispatch 2024-09-17T11:57:58.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:58 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:57:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[30017]: osdmap e746: 8 total, 8 up, 8 in 2024-09-17T11:57:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:59.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[30017]: pgmap v1152: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:57:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[40223]: osdmap e746: 8 total, 8 up, 8 in 2024-09-17T11:57:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:59.651 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:57:59 smithi049 ceph-mon[40223]: pgmap v1152: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:57:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:59 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "app": "rados"}]': finished 2024-09-17T11:57:59.676 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:59 smithi129 ceph-mon[33364]: osdmap e746: 8 total, 8 up, 8 in 2024-09-17T11:57:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:57:59.677 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:57:59 smithi129 ceph-mon[33364]: pgmap v1152: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:01 smithi049 ceph-mon[40223]: pgmap v1153: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-17T11:58:02.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:01 smithi049 ceph-mon[30017]: pgmap v1153: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-17T11:58:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:01 smithi129 ceph-mon[33364]: pgmap v1153: 176 pgs: 176 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 0 op/s 2024-09-17T11:58:03.677 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 11:58:03 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=sqlstore.transactions t=2024-09-17T11:58:03.422348036Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T11:58:04.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:03 smithi049 ceph-mon[30017]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-17T11:58:04.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:03 smithi049 ceph-mon[40223]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-17T11:58:04.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:03 smithi129 ceph-mon[33364]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-17T11:58:05.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[40223]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_objects: 10) 2024-09-17T11:58:05.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[40223]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:05.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[40223]: osdmap e747: 8 total, 8 up, 8 in 2024-09-17T11:58:05.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[30017]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_objects: 10) 2024-09-17T11:58:05.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[30017]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:05.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:04 smithi049 ceph-mon[30017]: osdmap e747: 8 total, 8 up, 8 in 2024-09-17T11:58:05.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:04 smithi129 ceph-mon[33364]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_objects: 10) 2024-09-17T11:58:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:04 smithi129 ceph-mon[33364]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:05.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:04 smithi129 ceph-mon[33364]: osdmap e747: 8 total, 8 up, 8 in 2024-09-17T11:58:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:05 smithi049 ceph-mon[40223]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-17T11:58:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:05 smithi049 ceph-mon[30017]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-17T11:58:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:05 smithi129 ceph-mon[33364]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-17T11:58:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:08.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:07 smithi049 ceph-mon[40223]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-17T11:58:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:07 smithi049 ceph-mon[30017]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-17T11:58:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:07 smithi129 ceph-mon[33364]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 601 B/s rd, 15 KiB/s wr, 1 op/s 2024-09-17T11:58:08.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:10.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:10.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:09 smithi049 ceph-mon[30017]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:09 smithi049 ceph-mon[40223]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:09 smithi129 ceph-mon[33364]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:12.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:11 smithi049 ceph-mon[40223]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:12.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:11 smithi049 ceph-mon[30017]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:11 smithi129 ceph-mon[33364]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:58:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[40223]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:14.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:14.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[30017]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:14.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:14.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:13 smithi129 ceph-mon[33364]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:13 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:14.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:16.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:15 smithi049 ceph-mon[40223]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-17T11:58:16.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:15 smithi049 ceph-mon[30017]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-17T11:58:16.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:15 smithi129 ceph-mon[33364]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-09-17T11:58:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:18.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:17 smithi049 ceph-mon[30017]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:18.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:17 smithi049 ceph-mon[40223]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:18.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:17 smithi129 ceph-mon[33364]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:18.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:19 smithi049 ceph-mon[40223]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:20.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:19 smithi049 ceph-mon[30017]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:19 smithi129 ceph-mon[33364]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:22.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:21 smithi049 ceph-mon[40223]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:21 smithi049 ceph-mon[30017]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:22.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:21 smithi129 ceph-mon[33364]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:23 smithi049 ceph-mon[40223]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:24.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:23 smithi049 ceph-mon[30017]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:23 smithi129 ceph-mon[33364]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:25 smithi049 ceph-mon[40223]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:25 smithi049 ceph-mon[30017]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:25 smithi129 ceph-mon[33364]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:28.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:27 smithi049 ceph-mon[30017]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:27 smithi049 ceph-mon[40223]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:28.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:28.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:27 smithi129 ceph-mon[33364]: pgmap v1167: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:28.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:28.655 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=145897 2024-09-17T11:58:28.655 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_objects 100 2024-09-17T11:58:28.656 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put onemore /etc/passwd 2024-09-17T11:58:28.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:28.756 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 -- 172.21.15.49:0/2678236038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c103890 msgr2=0x7f6e8c06a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/2678236038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c103890 0x7f6e8c06a2d0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009900 tx=0x7f6e7402a410 comp rx=0 tx=0).stop 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 -- 172.21.15.49:0/2678236038 shutdown_connections 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/2678236038 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 0x7f6e8c111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/2678236038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c06a810 0x7f6e8c06ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/2678236038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c103890 0x7f6e8c06a2d0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 -- 172.21.15.49:0/2678236038 >> 172.21.15.49:0/2678236038 conn(0x7f6e8c0776a0 msgr2=0x7f6e8c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 -- 172.21.15.49:0/2678236038 shutdown_connections 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.753+0000 7f6e92bb4640 1 -- 172.21.15.49:0/2678236038 wait complete. 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 Processor -- start 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 -- start start 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:28.757 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c103890 0x7f6e8c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 0x7f6e8c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1145e0 con 0x7f6e8c06a810 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c114460 con 0x7f6e8c103890 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e92bb4640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c114760 con 0x7f6e8c10f390 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52718/0 (socket says 172.21.15.49:52718) 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 -- 172.21.15.49:0/4023755842 learned_addr learned my addr 172.21.15.49:0/4023755842 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e8bfff640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c103890 0x7f6e8c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e91151640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 0x7f6e8c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 -- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 msgr2=0x7f6e8c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 0x7f6e8c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 -- 172.21.15.49:0/4023755842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c103890 msgr2=0x7f6e8c19b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:28.758 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c103890 0x7f6e8c19b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:28.759 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.754+0000 7f6e90950640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c1a0ba0 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e90950640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005940 tx=0x7f6e74032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7403f030 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f6e74005480 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c1a0e30 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74035070 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.755+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1a7550 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.756+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c06ac50 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.757+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f6e74042030 con 0x7f6e8c06a810 2024-09-17T11:58:28.760 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.758+0000 7f6e89ffb640 1 --2- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 0x7f6e7807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:28.761 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.758+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(747..747 src has 1..747) ==== 7758+0+0 (secure 0 0 0) 0x7f6e740be9f0 con 0x7f6e8c06a810 2024-09-17T11:58:28.761 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.758+0000 7f6e8bfff640 1 --2- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 0x7f6e7807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:28.761 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.758+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=748}) -- 0x7f6e78086220 con 0x7f6e8c06a810 2024-09-17T11:58:28.761 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.758+0000 7f6e8bfff640 1 --2- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 0x7f6e7807d290 secure :-1 s=READY pgs=3889 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c102bb0 tx=0x7f6e7c009680 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:58:28.764 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.762+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f6e74089b50 con 0x7f6e8c06a810 2024-09-17T11:58:28.958 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:28.956+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} v 0) -- 0x7f6e8c1a7720 con 0x7f6e8c06a810 2024-09-17T11:58:29.803 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:29.801+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v748) ==== 225+0+0 (secure 0 0 0) 0x7f6e7408ea00 con 0x7f6e8c06a810 2024-09-17T11:58:29.804 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:29.801+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 8 ==== osd_map(748..748 src has 1..748) ==== 628+0+0 (secure 0 0 0) 0x7f6e740bc830 con 0x7f6e8c06a810 2024-09-17T11:58:29.805 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:29.801+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=749}) -- 0x7f6e78087180 con 0x7f6e8c06a810 2024-09-17T11:58:29.922 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:29.919+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} v 0) -- 0x7f6e8c1a7970 con 0x7f6e8c06a810 2024-09-17T11:58:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[40223]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:30.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[30017]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:29 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:29 smithi129 ceph-mon[33364]: pgmap v1168: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:30.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:29 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:30.806 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.803+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v749) ==== 225+0+0 (secure 0 0 0) 0x7f6e74005630 con 0x7f6e8c06a810 2024-09-17T11:58:30.806 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_objects = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:58:30.806 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.803+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 <== mon.0 v2:172.21.15.49:3300/0 10 ==== osd_map(749..749 src has 1..749) ==== 628+0+0 (secure 0 0 0) 0x7f6e740bcb40 con 0x7f6e8c06a810 2024-09-17T11:58:30.806 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.803+0000 7f6e89ffb640 1 -- 172.21.15.49:0/4023755842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=750}) -- 0x7f6e78087180 con 0x7f6e8c06a810 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 msgr2=0x7f6e7807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 0x7f6e7807d290 secure :-1 s=READY pgs=3889 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c102bb0 tx=0x7f6e7c009680 comp rx=0 tx=0).stop 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 msgr2=0x7f6e8c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005940 tx=0x7f6e74032c70 comp rx=0 tx=0).stop 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 shutdown_connections 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> v2:172.21.15.49:6800/1163045599 conn(0x7f6e7807add0 0x7f6e7807d290 unknown :-1 s=CLOSED pgs=3889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f6e8c10f390 0x7f6e8c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:30.810 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e8c103890 0x7f6e8c19b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:30.811 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 --2- 172.21.15.49:0/4023755842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e8c06a810 0x7f6e8c19ab00 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:30.811 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.807+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 >> 172.21.15.49:0/4023755842 conn(0x7f6e8c0776a0 msgr2=0x7f6e8c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:58:30.811 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.808+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 shutdown_connections 2024-09-17T11:58:30.811 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:30.808+0000 7f6e92bb4640 1 -- 172.21.15.49:0/4023755842 wait complete. 2024-09-17T11:58:30.824 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 145897 2024-09-17T11:58:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:31.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[40223]: osdmap e748: 8 total, 8 up, 8 in 2024-09-17T11:58:31.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[30017]: osdmap e748: 8 total, 8 up, 8 in 2024-09-17T11:58:31.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:30 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:31.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:31.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:30 smithi129 ceph-mon[33364]: osdmap e748: 8 total, 8 up, 8 in 2024-09-17T11:58:31.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:30 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"} : dispatch 2024-09-17T11:58:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[40223]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[40223]: osdmap e749: 8 total, 8 up, 8 in 2024-09-17T11:58:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[30017]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:31 smithi049 ceph-mon[30017]: osdmap e749: 8 total, 8 up, 8 in 2024-09-17T11:58:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:31 smithi129 ceph-mon[33364]: pgmap v1170: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:31 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/4023755842' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "100"}]': finished 2024-09-17T11:58:32.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:31 smithi129 ceph-mon[33364]: osdmap e749: 8 total, 8 up, 8 in 2024-09-17T11:58:34.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:33 smithi049 ceph-mon[40223]: pgmap v1172: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:34.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:33 smithi049 ceph-mon[30017]: pgmap v1172: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:33 smithi129 ceph-mon[33364]: pgmap v1172: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:34.459 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 0 -ne 0 ']' 2024-09-17T11:58:34.459 INFO:tasks.workunit.client.0.smithi049.stderr:+ true 2024-09-17T11:58:34.460 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put twomore /etc/passwd 2024-09-17T11:58:34.500 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_bytes 100 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 -- 172.21.15.49:0/1872317560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc107f70 msgr2=0x7fcabc10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 --2- 172.21.15.49:0/1872317560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc107f70 0x7fcabc10e770 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab402a8c0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 -- 172.21.15.49:0/1872317560 shutdown_connections 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 --2- 172.21.15.49:0/1872317560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc107f70 0x7fcabc10e770 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 --2- 172.21.15.49:0/1872317560 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc1075f0 0x7fcabc107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 --2- 172.21.15.49:0/1872317560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc101bf0 0x7fcabc101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 -- 172.21.15.49:0/1872317560 >> 172.21.15.49:0/1872317560 conn(0x7fcabc0fd630 msgr2=0x7fcabc0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 -- 172.21.15.49:0/1872317560 shutdown_connections 2024-09-17T11:58:34.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.587+0000 7fcac2984640 1 -- 172.21.15.49:0/1872317560 wait complete. 2024-09-17T11:58:34.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 Processor -- start 2024-09-17T11:58:34.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 -- start start 2024-09-17T11:58:34.591 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:34.591 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 0x7fcabc198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc107f70 0x7fcabc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc111390 con 0x7fcabc101bf0 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc111210 con 0x7fcabc1075f0 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac2984640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fcabc111510 con 0x7fcabc107f70 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49266/0 (socket says 172.21.15.49:49266) 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcabbfff640 1 -- 172.21.15.49:0/1950779501 learned_addr learned my addr 172.21.15.49:0/1950779501 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcac0f21640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc107f70 0x7fcabc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcabb7fe640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 0x7fcabc198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.588+0000 7fcabbfff640 1 -- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc107f70 msgr2=0x7fcabc19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabbfff640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc107f70 0x7fcabc19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabbfff640 1 -- 172.21.15.49:0/1950779501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 msgr2=0x7fcabc198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:34.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabbfff640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 0x7fcabc198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabbfff640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19e8e0 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabb7fe640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 0x7fcabc198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcabbfff640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c920 tx=0x7fcaac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007c70 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fcaac007e10 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19ebd0 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018700 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.589+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a5290 con 0x7fcabc101bf0 2024-09-17T11:58:34.593 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.590+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc101fd0 con 0x7fcabc101bf0 2024-09-17T11:58:34.594 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.591+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fcaac00f450 con 0x7fcabc101bf0 2024-09-17T11:58:34.594 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.591+0000 7fcab97fa640 1 --2- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 0x7fcaa407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:58:34.594 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.591+0000 7fcabb7fe640 1 --2- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 0x7fcaa407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:58:34.594 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.591+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(750..750 src has 1..750) ==== 7758+0+0 (secure 0 0 0) 0x7fcaac09e6b0 con 0x7fcabc101bf0 2024-09-17T11:58:34.594 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.592+0000 7fcabb7fe640 1 --2- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 0x7fcaa407d290 secure :-1 s=READY pgs=3891 cs=0 l=1 rev1=1 crypto rx=0x7fcabc105bb0 tx=0x7fcab0005f90 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:58:34.597 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.596+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fcaac014030 con 0x7fcabc101bf0 2024-09-17T11:58:34.782 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:34.781+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fcabc1a5550 con 0x7fcabc101bf0 2024-09-17T11:58:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[30017]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:58:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[30017]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:58:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[30017]: osdmap e750: 8 total, 8 up, 8 in 2024-09-17T11:58:35.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[40223]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:58:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[40223]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:58:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[40223]: osdmap e750: 8 total, 8 up, 8 in 2024-09-17T11:58:35.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:34 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:34 smithi129 ceph-mon[33364]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:58:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:34 smithi129 ceph-mon[33364]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:58:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:34 smithi129 ceph-mon[33364]: osdmap e750: 8 total, 8 up, 8 in 2024-09-17T11:58:35.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:34 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:35.453 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:35.451+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v751) ==== 221+0+0 (secure 0 0 0) 0x7fcaac069810 con 0x7fcabc101bf0 2024-09-17T11:58:35.568 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:35.567+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fcabc1a57a0 con 0x7fcabc101bf0 2024-09-17T11:58:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[40223]: pgmap v1174: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[40223]: osdmap e751: 8 total, 8 up, 8 in 2024-09-17T11:58:36.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[30017]: pgmap v1174: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[30017]: osdmap e751: 8 total, 8 up, 8 in 2024-09-17T11:58:36.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:35 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:35 smithi129 ceph-mon[33364]: pgmap v1174: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:35 smithi129 ceph-mon[33364]: osdmap e751: 8 total, 8 up, 8 in 2024-09-17T11:58:36.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:35 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"} : dispatch 2024-09-17T11:58:36.454 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.453+0000 7fcab97fa640 1 -- 172.21.15.49:0/1950779501 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v752) ==== 221+0+0 (secure 0 0 0) 0x7fcaac06e6c0 con 0x7fcabc101bf0 2024-09-17T11:58:36.455 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_bytes = 100 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 msgr2=0x7fcaa407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 0x7fcaa407d290 secure :-1 s=READY pgs=3891 cs=0 l=1 rev1=1 crypto rx=0x7fcabc105bb0 tx=0x7fcab0005f90 comp rx=0 tx=0).stop 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 msgr2=0x7fcabc198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c920 tx=0x7fcaac00cde0 comp rx=0 tx=0).stop 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 shutdown_connections 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> v2:172.21.15.49:6800/1163045599 conn(0x7fcaa407add0 0x7fcaa407d290 unknown :-1 s=CLOSED pgs=3891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcabc107f70 0x7fcabc19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:36.459 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcabc1075f0 0x7fcabc198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:36.460 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 --2- 172.21.15.49:0/1950779501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcabc101bf0 0x7fcabc198840 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:58:36.460 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 >> 172.21.15.49:0/1950779501 conn(0x7fcabc0fd630 msgr2=0x7fcabc06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:58:36.460 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 shutdown_connections 2024-09-17T11:58:36.460 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:58:36.457+0000 7fcac2984640 1 -- 172.21.15.49:0/1950779501 wait complete. 2024-09-17T11:58:36.470 INFO:tasks.workunit.client.0.smithi049.stderr:+ sleep 30 2024-09-17T11:58:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:37.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:37.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[40223]: osdmap e752: 8 total, 8 up, 8 in 2024-09-17T11:58:37.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[40223]: pgmap v1177: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-09-17T11:58:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[30017]: osdmap e752: 8 total, 8 up, 8 in 2024-09-17T11:58:37.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:37 smithi049 ceph-mon[30017]: pgmap v1177: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-09-17T11:58:37.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:37 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1950779501' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "100"}]': finished 2024-09-17T11:58:37.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:37 smithi129 ceph-mon[33364]: osdmap e752: 8 total, 8 up, 8 in 2024-09-17T11:58:37.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:37 smithi129 ceph-mon[33364]: pgmap v1177: 176 pgs: 176 active+clean; 708 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-09-17T11:58:38.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[40223]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[40223]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_bytes: 100 B) 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[40223]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[40223]: osdmap e753: 8 total, 8 up, 8 in 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[30017]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[30017]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_bytes: 100 B) 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[30017]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:39 smithi049 ceph-mon[30017]: osdmap e753: 8 total, 8 up, 8 in 2024-09-17T11:58:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:40.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:39 smithi129 ceph-mon[33364]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:39 smithi129 ceph-mon[33364]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' is full (reached quota's max_bytes: 100 B) 2024-09-17T11:58:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:39 smithi129 ceph-mon[33364]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:58:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:39 smithi129 ceph-mon[33364]: osdmap e753: 8 total, 8 up, 8 in 2024-09-17T11:58:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:41 smithi049 ceph-mon[40223]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:42.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:41 smithi049 ceph-mon[30017]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:41 smithi129 ceph-mon[33364]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-17T11:58:43.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:43.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:43.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:43 smithi049 ceph-mon[40223]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-17T11:58:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:43 smithi049 ceph-mon[30017]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-17T11:58:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:43 smithi129 ceph-mon[33364]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-17T11:58:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:45 smithi049 ceph-mon[40223]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-17T11:58:45.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:45 smithi049 ceph-mon[30017]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-17T11:58:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:45 smithi129 ceph-mon[33364]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-17T11:58:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:48.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:47 smithi049 ceph-mon[40223]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-17T11:58:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:47 smithi049 ceph-mon[30017]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-17T11:58:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:47 smithi129 ceph-mon[33364]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-17T11:58:48.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:49 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:49 smithi049 ceph-mon[40223]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:49 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:49 smithi049 ceph-mon[30017]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:49 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:58:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:49 smithi129 ceph-mon[33364]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T11:58:52.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:51 smithi049 ceph-mon[40223]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-17T11:58:52.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:51 smithi049 ceph-mon[30017]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-17T11:58:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:51 smithi129 ceph-mon[33364]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-17T11:58:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:53 smithi049 ceph-mon[40223]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:53 smithi049 ceph-mon[30017]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:53 smithi129 ceph-mon[33364]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:58:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:55 smithi049 ceph-mon[40223]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:56.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:55 smithi049 ceph-mon[30017]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:55 smithi129 ceph-mon[33364]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:58:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:58:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:58:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:58:56.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:58:56.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:58:56.902 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:56 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:58:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:58:57.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:58:57.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:56 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:58:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:57 smithi049 ceph-mon[40223]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:58.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:57 smithi049 ceph-mon[30017]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:57 smithi129 ceph-mon[33364]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:58:58.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:58:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:58:59.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:58 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:58 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:58:59.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:58 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:58:59 smithi049 ceph-mon[40223]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:58:59 smithi049 ceph-mon[30017]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:58:59 smithi129 ceph-mon[33364]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:01 smithi049 ceph-mon[40223]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:59:02.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:01 smithi049 ceph-mon[30017]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:59:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:01 smithi129 ceph-mon[33364]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T11:59:04.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:03 smithi049 ceph-mon[40223]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:04.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:03 smithi049 ceph-mon[30017]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:04.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:03 smithi129 ceph-mon[33364]: pgmap v1191: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:06.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:05 smithi049 ceph-mon[40223]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:06.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:05 smithi049 ceph-mon[30017]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:05 smithi129 ceph-mon[33364]: pgmap v1192: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:06.472 INFO:tasks.workunit.client.0.smithi049.stderr:+ pid=146065 2024-09-17T11:59:06.473 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_bytes 0 2024-09-17T11:59:06.473 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put two /etc/passwd 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- 172.21.15.49:0/2371338268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 msgr2=0x7f3898108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 --2- 172.21.15.49:0/2371338268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898108b30 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f388c009900 tx=0x7f388c02a410 comp rx=0 tx=0).stop 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- 172.21.15.49:0/2371338268 shutdown_connections 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 --2- 172.21.15.49:0/2371338268 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 --2- 172.21.15.49:0/2371338268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898108b30 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 --2- 172.21.15.49:0/2371338268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 0x7f38981030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- 172.21.15.49:0/2371338268 >> 172.21.15.49:0/2371338268 conn(0x7f38980fe750 msgr2=0x7f3898100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- 172.21.15.49:0/2371338268 shutdown_connections 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- 172.21.15.49:0/2371338268 wait complete. 2024-09-17T11:59:06.572 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 Processor -- start 2024-09-17T11:59:06.573 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.569+0000 7f389e439640 1 -- start start 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 0x7f3898198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3898112440 con 0x7f38981086f0 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38981122c0 con 0x7f3898102cf0 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389e439640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f38981125c0 con 0x7f3898109070 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f38977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f38977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38708/0 (socket says 172.21.15.49:38708) 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f38977fe640 1 -- 172.21.15.49:0/3036147720 learned_addr learned my addr 172.21.15.49:0/3036147720 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f3897fff640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 0x7f3898198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 -- 172.21.15.49:0/3036147720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 msgr2=0x7f3898198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:06.575 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 0x7f3898198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 -- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 msgr2=0x7f3898198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f389c9d6640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389819e9e0 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.570+0000 7f38977fe640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f389c9d6640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389819e210 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f389000ec50 tx=0x7f389000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890003b70 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f3890003d10 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389819ecd0 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890010650 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.571+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a5390 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.572+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38981030d0 con 0x7f3898109070 2024-09-17T11:59:06.576 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.573+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f3890004020 con 0x7f3898109070 2024-09-17T11:59:06.577 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.573+0000 7f38957fa640 1 --2- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 0x7f388007d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:06.577 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.573+0000 7f3897fff640 1 --2- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 0x7f388007d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:06.577 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.573+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(753..753 src has 1..753) ==== 7758+0+0 (secure 0 0 0) 0x7f389009ea40 con 0x7f3898109070 2024-09-17T11:59:06.577 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.573+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=754}) -- 0x7f3880086220 con 0x7f3898109070 2024-09-17T11:59:06.577 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.574+0000 7f3897fff640 1 --2- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 0x7f388007d290 secure :-1 s=READY pgs=3893 cs=0 l=1 rev1=1 crypto rx=0x7f3888005d00 tx=0x7f388800b410 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:06.580 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.578+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f3890069ba0 con 0x7f3898109070 2024-09-17T11:59:06.770 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.765+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f389810ae50 con 0x7f3898109070 2024-09-17T11:59:06.846 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.844+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 7 ==== osd_map(754..754 src has 1..754) ==== 628+0+0 (secure 0 0 0) 0x7f389009c880 con 0x7f3898109070 2024-09-17T11:59:06.846 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.844+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=755}) -- 0x7f3880086d90 con 0x7f3898109070 2024-09-17T11:59:06.849 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.847+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v754) ==== 217+0+0 (secure 0 0 0) 0x7f389009ee20 con 0x7f3898109070 2024-09-17T11:59:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:06 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:06.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:06 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:06 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:06.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:06 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:06.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:06.961 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:06.958+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f389810a8c0 con 0x7f3898109070 2024-09-17T11:59:07.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:06 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:07.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:06 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:07.853 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.850+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 9 ==== osd_map(755..755 src has 1..755) ==== 628+0+0 (secure 0 0 0) 0x7f38900613e0 con 0x7f3898109070 2024-09-17T11:59:07.853 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.850+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=756}) -- 0x7f3880087180 con 0x7f3898109070 2024-09-17T11:59:07.858 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.856+0000 7f38957fa640 1 -- 172.21.15.49:0/3036147720 <== mon.2 v2:172.21.15.49:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v755) ==== 217+0+0 (secure 0 0 0) 0x7f389006c0c0 con 0x7f3898109070 2024-09-17T11:59:07.858 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.860+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 msgr2=0x7f388007d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.860+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 0x7f388007d290 secure :-1 s=READY pgs=3893 cs=0 l=1 rev1=1 crypto rx=0x7f3888005d00 tx=0x7f388800b410 comp rx=0 tx=0).stop 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.860+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 msgr2=0x7f389819e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.860+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389819e210 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f389000ec50 tx=0x7f389000c6a0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 shutdown_connections 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> v2:172.21.15.49:6800/1163045599 conn(0x7f388007add0 0x7f388007d290 unknown :-1 s=CLOSED pgs=3893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f3898109070 0x7f389819e210 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38981086f0 0x7f3898198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.863 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 --2- 172.21.15.49:0/3036147720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3898102cf0 0x7f3898198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.864 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 >> 172.21.15.49:0/3036147720 conn(0x7f38980fe750 msgr2=0x7f3898100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:07.864 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 shutdown_connections 2024-09-17T11:59:07.864 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.861+0000 7f389e439640 1 -- 172.21.15.49:0/3036147720 wait complete. 2024-09-17T11:59:07.879 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_objects 0 2024-09-17T11:59:07.969 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 -- 172.21.15.49:0/3141816116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 msgr2=0x7f55c0103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:07.969 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 --2- 172.21.15.49:0/3141816116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0103fe0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009860 tx=0x7f55b002a340 comp rx=0 tx=0).stop 2024-09-17T11:59:07.969 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 -- 172.21.15.49:0/3141816116 shutdown_connections 2024-09-17T11:59:07.969 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 --2- 172.21.15.49:0/3141816116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c010d050 0x7f55c010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 --2- 172.21.15.49:0/3141816116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c0104520 0x7f55c010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 --2- 172.21.15.49:0/3141816116 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0103fe0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 -- 172.21.15.49:0/3141816116 >> 172.21.15.49:0/3141816116 conn(0x7f55c00fd630 msgr2=0x7f55c00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 -- 172.21.15.49:0/3141816116 shutdown_connections 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.966+0000 7f55c4d91640 1 -- 172.21.15.49:0/3141816116 wait complete. 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 Processor -- start 2024-09-17T11:59:07.970 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 -- start start 2024-09-17T11:59:07.973 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:07.973 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0104520 0x7f55c0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:07.973 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 0x7f55c019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:07.973 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01123f0 con 0x7f55c010d050 2024-09-17T11:59:07.973 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0112270 con 0x7f55c0104520 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55c4d91640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f55c0112570 con 0x7f55c0103c00 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55bed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 0x7f55c019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.967+0000 7f55be59c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:57974/0 (socket says 172.21.15.49:57974) 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 -- 172.21.15.49:0/587654727 learned_addr learned my addr 172.21.15.49:0/587654727 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55bdd9b640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0104520 0x7f55c0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 -- 172.21.15.49:0/587654727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0104520 msgr2=0x7f55c0198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0104520 0x7f55c0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 -- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 msgr2=0x7f55c019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 0x7f55c019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c019eac0 con 0x7f55c0103c00 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55bed9d640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 0x7f55c019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:59:07.974 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55be59c640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009980 tx=0x7f55b0033d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b003f020 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f55b00044c0 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0037040 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c019ed50 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.968+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c01a5470 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.969+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c0103fe0 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.970+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f55b0042020 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.970+0000 7f55af7fe640 1 --2- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 0x7f55a807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.971+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(755..755 src has 1..755) ==== 7758+0+0 (secure 0 0 0) 0x7f55b00be590 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.971+0000 7f55bdd9b640 1 --2- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 0x7f55a807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.971+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=756}) -- 0x7f55a8086220 con 0x7f55c0103c00 2024-09-17T11:59:07.975 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.971+0000 7f55bdd9b640 1 --2- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 0x7f55a807d290 secure :-1 s=READY pgs=3894 cs=0 l=1 rev1=1 crypto rx=0x7f55c010e090 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:07.977 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:07.976+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f55b0049020 con 0x7f55c0103c00 2024-09-17T11:59:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[30017]: pgmap v1193: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[30017]: osdmap e754: 8 total, 8 up, 8 in 2024-09-17T11:59:08.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[40223]: pgmap v1193: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[40223]: osdmap e754: 8 total, 8 up, 8 in 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:07 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.165 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:08.163+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} v 0) -- 0x7f55c0107790 con 0x7f55c0103c00 2024-09-17T11:59:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:07 smithi129 ceph-mon[33364]: pgmap v1193: 176 pgs: 176 active+clean; 733 KiB data, 812 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:07 smithi129 ceph-mon[33364]: osdmap e754: 8 total, 8 up, 8 in 2024-09-17T11:59:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:07 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3036147720' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:07 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T11:59:08.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:08.860 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:08.858+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 7 ==== osd_map(756..756 src has 1..756) ==== 628+0+0 (secure 0 0 0) 0x7f55b00f5190 con 0x7f55c0103c00 2024-09-17T11:59:08.860 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:08.858+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=757}) -- 0x7f55a8086d90 con 0x7f55c0103c00 2024-09-17T11:59:08.864 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:08.863+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v756) ==== 221+0+0 (secure 0 0 0) 0x7f55b00896f0 con 0x7f55c0103c00 2024-09-17T11:59:08.978 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:08.977+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} v 0) -- 0x7f55c01073a0 con 0x7f55c0103c00 2024-09-17T11:59:09.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:09.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[30017]: osdmap e755: 8 total, 8 up, 8 in 2024-09-17T11:59:09.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:09.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[40223]: osdmap e755: 8 total, 8 up, 8 in 2024-09-17T11:59:09.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:08 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T11:59:09.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:08 smithi129 ceph-mon[33364]: osdmap e755: 8 total, 8 up, 8 in 2024-09-17T11:59:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:08 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:08 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:09.457 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.455+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 9 ==== osd_map(757..757 src has 1..757) ==== 628+0+0 (secure 0 0 0) 0x7f55b00bc3d0 con 0x7f55c0103c00 2024-09-17T11:59:09.457 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.455+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=758}) -- 0x7f55a8087180 con 0x7f55c0103c00 2024-09-17T11:59:09.462 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.460+0000 7f55af7fe640 1 -- 172.21.15.49:0/587654727 <== mon.2 v2:172.21.15.49:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v757) ==== 221+0+0 (secure 0 0 0) 0x7f55b008e5a0 con 0x7f55c0103c00 2024-09-17T11:59:09.463 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T11:59:09.467 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 msgr2=0x7f55a807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:09.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 0x7f55a807d290 secure :-1 s=READY pgs=3894 cs=0 l=1 rev1=1 crypto rx=0x7f55c010e090 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2024-09-17T11:59:09.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 msgr2=0x7f55c0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:09.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009980 tx=0x7f55b0033d30 comp rx=0 tx=0).stop 2024-09-17T11:59:09.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 shutdown_connections 2024-09-17T11:59:09.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> v2:172.21.15.49:6800/1163045599 conn(0x7f55a807add0 0x7f55a807d290 unknown :-1 s=CLOSED pgs=3894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.469 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55c010d050 0x7f55c019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.469 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0104520 0x7f55c0198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.469 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 --2- 172.21.15.49:0/587654727 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f55c0103c00 0x7f55c0198a20 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.469 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.465+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 >> 172.21.15.49:0/587654727 conn(0x7f55c00fd630 msgr2=0x7f55c0104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:09.469 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.466+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 shutdown_connections 2024-09-17T11:59:09.470 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.466+0000 7f55c4d91640 1 -- 172.21.15.49:0/587654727 wait complete. 2024-09-17T11:59:09.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ wait 146065 2024-09-17T11:59:09.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ '[' 0 -ne 0 ']' 2024-09-17T11:59:09.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ true 2024-09-17T11:59:09.480 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put three /etc/passwd 2024-09-17T11:59:09.519 INFO:tasks.workunit.client.0.smithi049.stderr:++ uuidgen 2024-09-17T11:59:09.520 INFO:tasks.workunit.client.0.smithi049.stderr:+ pp=2c90288b-f52d-4a05-8035-9647849d2552 2024-09-17T11:59:09.520 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool create 2c90288b-f52d-4a05-8035-9647849d2552 12 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- 172.21.15.49:0/46636923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c107f70 msgr2=0x7f074c10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 --2- 172.21.15.49:0/46636923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c107f70 0x7f074c10e770 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f0744009be0 tx=0x7f074402a6c0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- 172.21.15.49:0/46636923 shutdown_connections 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 --2- 172.21.15.49:0/46636923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c107f70 0x7f074c10e770 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 --2- 172.21.15.49:0/46636923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c1075f0 0x7f074c107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.611 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 --2- 172.21.15.49:0/46636923 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c101bf0 0x7f074c101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- 172.21.15.49:0/46636923 >> 172.21.15.49:0/46636923 conn(0x7f074c0fd630 msgr2=0x7f074c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- 172.21.15.49:0/46636923 shutdown_connections 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- 172.21.15.49:0/46636923 wait complete. 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 Processor -- start 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.606+0000 7f07512f6640 1 -- start start 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 0x7f074c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:09.612 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f074c111300 con 0x7f074c1075f0 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f074c111180 con 0x7f074c101bf0 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f074c111480 con 0x7f074c107f70 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074b7fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38786/0 (socket says 172.21.15.49:38786) 2024-09-17T11:59:09.613 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074b7fe640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:58020/0 (socket says 172.21.15.49:58020) 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 -- 172.21.15.49:0/3571327917 learned_addr learned my addr 172.21.15.49:0/3571327917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074affd640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 0x7f074c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 -- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 msgr2=0x7f074c19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 -- 172.21.15.49:0/3571327917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 msgr2=0x7f074c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:09.614 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 0x7f074c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c19d840 con 0x7f074c1075f0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074b7fe640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074affd640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 0x7f074c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f074a7fc640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f0740007910 tx=0x7f0740007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740019070 con 0x7f074c1075f0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f0740005020 con 0x7f074c1075f0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.607+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c19db30 con 0x7f074c1075f0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.608+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400044a0 con 0x7f074c1075f0 2024-09-17T11:59:09.615 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.608+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c06b2a0 con 0x7f074c1075f0 2024-09-17T11:59:09.616 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.608+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c101fd0 con 0x7f074c1075f0 2024-09-17T11:59:09.616 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.609+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f074001c070 con 0x7f074c1075f0 2024-09-17T11:59:09.617 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.610+0000 7f0733fff640 1 --2- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 0x7f073407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:09.617 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.610+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(757..757 src has 1..757) ==== 7758+0+0 (secure 0 0 0) 0x7f074009f370 con 0x7f074c1075f0 2024-09-17T11:59:09.617 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.610+0000 7f074affd640 1 --2- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 0x7f073407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:09.617 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.610+0000 7f074affd640 1 --2- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 0x7f073407d290 secure :-1 s=READY pgs=3896 cs=0 l=1 rev1=1 crypto rx=0x7f073c005fd0 tx=0x7f073c009260 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:09.617 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.614+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f074006a4d0 con 0x7f074c1075f0 2024-09-17T11:59:09.804 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:09.803+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} v 0) -- 0x7f074c06bcc0 con 0x7f074c1075f0 2024-09-17T11:59:10.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: osdmap e756: 8 total, 8 up, 8 in 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: osdmap e757: 8 total, 8 up, 8 in 2024-09-17T11:59:10.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: osdmap e756: 8 total, 8 up, 8 in 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: osdmap e757: 8 total, 8 up, 8 in 2024-09-17T11:59:10.153 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:09 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: pgmap v1196: 176 pgs: 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: osdmap e756: 8 total, 8 up, 8 in 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/587654727' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T11:59:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' no longer out of quota; removing NO_QUOTA flag 2024-09-17T11:59:10.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T11:59:10.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T11:59:10.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: osdmap e757: 8 total, 8 up, 8 in 2024-09-17T11:59:10.178 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:09 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:10.458 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.456+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12}]=0 pool '2c90288b-f52d-4a05-8035-9647849d2552' created v758) ==== 176+0+0 (secure 0 0 0) 0x7f074006f380 con 0x7f074c1075f0 2024-09-17T11:59:10.580 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.578+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} v 0) -- 0x7f074c06bfb0 con 0x7f074c1075f0 2024-09-17T11:59:10.580 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.579+0000 7f0733fff640 1 -- 172.21.15.49:0/3571327917 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12}]=0 pool '2c90288b-f52d-4a05-8035-9647849d2552' already exists v758) ==== 183+0+0 (secure 0 0 0) 0x7f074009d1b0 con 0x7f074c1075f0 2024-09-17T11:59:10.580 INFO:tasks.workunit.client.0.smithi049.stderr:pool '2c90288b-f52d-4a05-8035-9647849d2552' already exists 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 msgr2=0x7f073407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 0x7f073407d290 secure :-1 s=READY pgs=3896 cs=0 l=1 rev1=1 crypto rx=0x7f073c005fd0 tx=0x7f073c009260 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 msgr2=0x7f074c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f0740007910 tx=0x7f0740007dd0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 shutdown_connections 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> v2:172.21.15.49:6800/1163045599 conn(0x7f073407add0 0x7f073407d290 unknown :-1 s=CLOSED pgs=3896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f074c107f70 0x7f074c19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f074c1075f0 0x7f074c197ce0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 --2- 172.21.15.49:0/3571327917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f074c101bf0 0x7f074c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 >> 172.21.15.49:0/3571327917 conn(0x7f074c0fd630 msgr2=0x7f074c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:10.583 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 shutdown_connections 2024-09-17T11:59:10.584 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.581+0000 7f07512f6640 1 -- 172.21.15.49:0/3571327917 wait complete. 2024-09-17T11:59:10.592 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool application enable 2c90288b-f52d-4a05-8035-9647849d2552 rados 2024-09-17T11:59:10.679 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 -- 172.21.15.49:0/313579918 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075e60 msgr2=0x7fcccc113510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 --2- 172.21.15.49:0/313579918 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075e60 0x7fcccc113510 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009900 tx=0x7fccbc02a410 comp rx=0 tx=0).stop 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 -- 172.21.15.49:0/313579918 shutdown_connections 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 --2- 172.21.15.49:0/313579918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 0x7fcccc115e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 --2- 172.21.15.49:0/313579918 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075e60 0x7fcccc113510 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 --2- 172.21.15.49:0/313579918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcccc075540 0x7fcccc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.676+0000 7fccd3860640 1 -- 172.21.15.49:0/313579918 >> 172.21.15.49:0/313579918 conn(0x7fcccc0fe740 msgr2=0x7fcccc100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- 172.21.15.49:0/313579918 shutdown_connections 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- 172.21.15.49:0/313579918 wait complete. 2024-09-17T11:59:10.680 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 Processor -- start 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- start start 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcccc075e60 0x7fcccc19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 0x7fcccc1a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc118980 con 0x7fcccc113a50 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc118800 con 0x7fcccc075e60 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd3860640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc118b00 con 0x7fcccc075540 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd15fc640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd15fc640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:58034/0 (socket says 172.21.15.49:58034) 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 -- 172.21.15.49:0/849990325 learned_addr learned my addr 172.21.15.49:0/849990325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.677+0000 7fccd1dfd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 0x7fcccc1a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 -- 172.21.15.49:0/849990325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcccc075e60 msgr2=0x7fcccc19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:10.681 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcccc075e60 0x7fcccc19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 -- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 msgr2=0x7fcccc1a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 0x7fcccc1a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc1a5070 con 0x7fcccc075540 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd15fc640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7fccc800d950 tx=0x7fccc800dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8017070 con 0x7fcccc075540 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fccc800de80 con 0x7fcccc075540 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc1a5360 con 0x7fcccc075540 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8010e60 con 0x7fcccc075540 2024-09-17T11:59:10.682 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.678+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc1aba20 con 0x7fcccc075540 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.679+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc075920 con 0x7fcccc075540 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.680+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fccc8010460 con 0x7fcccc075540 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.680+0000 7fccc27fc640 1 --2- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 0x7fccb807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.680+0000 7fccd0dfb640 1 --2- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 0x7fccb807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.680+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(758..758 src has 1..758) ==== 8133+0+0 (secure 0 0 0) 0x7fccc809e2f0 con 0x7fcccc075540 2024-09-17T11:59:10.683 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.681+0000 7fccd0dfb640 1 --2- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 0x7fccb807d290 secure :-1 s=READY pgs=3897 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0096c0 tx=0x7fccbc031400 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:10.686 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.685+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fccc80692d0 con 0x7fcccc075540 2024-09-17T11:59:10.877 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:10.875+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} v 0) -- 0x7fcccc1abc80 con 0x7fcccc075540 2024-09-17T11:59:11.468 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:11.466+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]=0 enabled application 'rados' on pool '2c90288b-f52d-4a05-8035-9647849d2552' v759) ==== 213+0+0 (secure 0 0 0) 0x7fccc806e180 con 0x7fcccc075540 2024-09-17T11:59:11.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:11.589+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} v 0) -- 0x7fcccc10f690 con 0x7fcccc075540 2024-09-17T11:59:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12}]': finished 2024-09-17T11:59:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: osdmap e758: 8 total, 8 up, 8 in 2024-09-17T11:59:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:11.900 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: pgmap v1200: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12}]': finished 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: osdmap e758: 8 total, 8 up, 8 in 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: pgmap v1200: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:11.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:11 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12}]': finished 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: osdmap e758: 8 total, 8 up, 8 in 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3571327917' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pg_num": 12} : dispatch 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: pgmap v1200: 188 pgs: 12 unknown, 176 active+clean; 733 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:11.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:11 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.477 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.475+0000 7fccc27fc640 1 -- 172.21.15.49:0/849990325 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]=0 enabled application 'rados' on pool '2c90288b-f52d-4a05-8035-9647849d2552' v760) ==== 213+0+0 (secure 0 0 0) 0x7fccc809bfb0 con 0x7fcccc075540 2024-09-17T11:59:12.477 INFO:tasks.workunit.client.0.smithi049.stderr:enabled application 'rados' on pool '2c90288b-f52d-4a05-8035-9647849d2552' 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 msgr2=0x7fccb807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 0x7fccb807d290 secure :-1 s=READY pgs=3897 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0096c0 tx=0x7fccbc031400 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 msgr2=0x7fcccc19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7fccc800d950 tx=0x7fccc800dd50 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 shutdown_connections 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> v2:172.21.15.49:6800/1163045599 conn(0x7fccb807add0 0x7fccb807d290 unknown :-1 s=CLOSED pgs=3897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcccc113a50 0x7fcccc1a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcccc075e60 0x7fcccc19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 --2- 172.21.15.49:0/849990325 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fcccc075540 0x7fcccc19efd0 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.480 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 >> 172.21.15.49:0/849990325 conn(0x7fcccc0fe740 msgr2=0x7fcccc105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:12.481 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.478+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 shutdown_connections 2024-09-17T11:59:12.481 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.479+0000 7fccd3860640 1 -- 172.21.15.49:0/849990325 wait complete. 2024-09-17T11:59:12.491 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota 2c90288b-f52d-4a05-8035-9647849d2552 max_objects 10 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 -- 172.21.15.49:0/1438516039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7384103a70 msgr2=0x7f7384103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 --2- 172.21.15.49:0/1438516039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7384103a70 0x7f7384103e50 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f7380009900 tx=0x7f738002a410 comp rx=0 tx=0).stop 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 -- 172.21.15.49:0/1438516039 shutdown_connections 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 --2- 172.21.15.49:0/1438516039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f738410d020 0x7f738410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 --2- 172.21.15.49:0/1438516039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f738410ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 --2- 172.21.15.49:0/1438516039 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f7384103a70 0x7f7384103e50 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.586 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 -- 172.21.15.49:0/1438516039 >> 172.21.15.49:0/1438516039 conn(0x7f73840fd640 msgr2=0x7f73840ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 -- 172.21.15.49:0/1438516039 shutdown_connections 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.582+0000 7f738c764640 1 -- 172.21.15.49:0/1438516039 wait complete. 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 Processor -- start 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 -- start start 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f7384101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73841123c0 con 0x7f7384103a70 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7384112240 con 0x7f7384104470 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738c764640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f7384112540 con 0x7f738410d020 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738ad01640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:12.587 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738a500640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738ad01640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:58042/0 (socket says 172.21.15.49:58042) 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738a500640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38834/0 (socket says 172.21.15.49:38834) 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.583+0000 7f738ad01640 1 -- 172.21.15.49:0/1856503950 learned_addr learned my addr 172.21.15.49:0/1856503950 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f7389cff640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f7384101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 -- 172.21.15.49:0/1856503950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 msgr2=0x7f7384101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f7384101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 -- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 msgr2=0x7f73841018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73841a6df0 con 0x7f738410d020 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f7389cff640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f7384101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738a500640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738ad01640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f737c00eb10 tx=0x7f737c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:12.588 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c004b20 con 0x7f738410d020 2024-09-17T11:59:12.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f737c004cc0 con 0x7f738410d020 2024-09-17T11:59:12.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73841a70e0 con 0x7f738410d020 2024-09-17T11:59:12.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c010690 con 0x7f738410d020 2024-09-17T11:59:12.589 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.584+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73841a7630 con 0x7f738410d020 2024-09-17T11:59:12.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.585+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7384103e50 con 0x7f738410d020 2024-09-17T11:59:12.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.586+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f737c005020 con 0x7f738410d020 2024-09-17T11:59:12.590 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.586+0000 7f737b7fe640 1 --2- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 0x7f737007d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T11:59:12.591 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.586+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 5 ==== osd_map(760..760 src has 1..760) ==== 8146+0+0 (secure 0 0 0) 0x7f737c09ec00 con 0x7f738410d020 2024-09-17T11:59:12.591 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.586+0000 7f738a500640 1 --2- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 0x7f737007d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T11:59:12.591 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.587+0000 7f738a500640 1 --2- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 0x7f737007d290 secure :-1 s=READY pgs=3898 cs=0 l=1 rev1=1 crypto rx=0x7f7380031440 tx=0x7f7380031850 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T11:59:12.592 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.591+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f737c0a5080 con 0x7f738410d020 2024-09-17T11:59:12.779 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:12.778+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} v 0) -- 0x7f7384107820 con 0x7f738410d020 2024-09-17T11:59:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[40223]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:12.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[40223]: osdmap e759: 8 total, 8 up, 8 in 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[30017]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[30017]: osdmap e759: 8 total, 8 up, 8 in 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:12 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:12 smithi129 ceph-mon[33364]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:12 smithi129 ceph-mon[33364]: osdmap e759: 8 total, 8 up, 8 in 2024-09-17T11:59:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:12 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/849990325' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:12.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:12 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"} : dispatch 2024-09-17T11:59:13.483 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:13.481+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 2c90288b-f52d-4a05-8035-9647849d2552 v761) ==== 223+0+0 (secure 0 0 0) 0x7f737c069c50 con 0x7f738410d020 2024-09-17T11:59:13.597 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:13.596+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} v 0) -- 0x7f7384107290 con 0x7f738410d020 2024-09-17T11:59:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: osdmap e760: 8 total, 8 up, 8 in 2024-09-17T11:59:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-17T11:59:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: osdmap e760: 8 total, 8 up, 8 in 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:13.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:13 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "app": "rados"}]': finished 2024-09-17T11:59:13.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: osdmap e760: 8 total, 8 up, 8 in 2024-09-17T11:59:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: pgmap v1203: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-17T11:59:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:13.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:13 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:14.487 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.485+0000 7f737b7fe640 1 -- 172.21.15.49:0/1856503950 <== mon.2 v2:172.21.15.49:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 2c90288b-f52d-4a05-8035-9647849d2552 v762) ==== 223+0+0 (secure 0 0 0) 0x7f737c06eb00 con 0x7f738410d020 2024-09-17T11:59:14.487 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_objects = 10 for pool 2c90288b-f52d-4a05-8035-9647849d2552 2024-09-17T11:59:14.490 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 msgr2=0x7f737007d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 0x7f737007d290 secure :-1 s=READY pgs=3898 cs=0 l=1 rev1=1 crypto rx=0x7f7380031440 tx=0x7f7380031850 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 msgr2=0x7f7384102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f737c00eb10 tx=0x7f737c00c2d0 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 shutdown_connections 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> v2:172.21.15.49:6800/1163045599 conn(0x7f737007add0 0x7f737007d290 unknown :-1 s=CLOSED pgs=3898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f738410d020 0x7f7384102370 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7384104470 0x7f7384101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 --2- 172.21.15.49:0/1856503950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7384103a70 0x7f73841018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 >> 172.21.15.49:0/1856503950 conn(0x7f73840fd640 msgr2=0x7f73841049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 shutdown_connections 2024-09-17T11:59:14.491 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T11:59:14.489+0000 7f738c764640 1 -- 172.21.15.49:0/1856503950 wait complete. 2024-09-17T11:59:14.501 INFO:tasks.workunit.client.0.smithi049.stderr:+ sleep 30 2024-09-17T11:59:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[40223]: osdmap e761: 8 total, 8 up, 8 in 2024-09-17T11:59:14.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:14.901 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[30017]: osdmap e761: 8 total, 8 up, 8 in 2024-09-17T11:59:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:14.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:14 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:14.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:14.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:14 smithi129 ceph-mon[33364]: osdmap e761: 8 total, 8 up, 8 in 2024-09-17T11:59:14.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:14 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1856503950' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:14.928 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:14 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"} : dispatch 2024-09-17T11:59:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[40223]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[40223]: osdmap e762: 8 total, 8 up, 8 in 2024-09-17T11:59:15.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[40223]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-17T11:59:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[30017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[30017]: osdmap e762: 8 total, 8 up, 8 in 2024-09-17T11:59:15.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:15 smithi049 ceph-mon[30017]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-17T11:59:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:15 smithi129 ceph-mon[33364]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "field": "max_objects", "val": "10"}]': finished 2024-09-17T11:59:15.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:15 smithi129 ceph-mon[33364]: osdmap e762: 8 total, 8 up, 8 in 2024-09-17T11:59:15.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:15 smithi129 ceph-mon[33364]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-17T11:59:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:18.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:17 smithi049 ceph-mon[40223]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-17T11:59:18.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:17 smithi049 ceph-mon[30017]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-17T11:59:18.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:17 smithi129 ceph-mon[33364]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-17T11:59:18.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[40223]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:59:20.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[40223]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[30017]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:59:20.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:19 smithi049 ceph-mon[30017]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:20.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:19 smithi129 ceph-mon[33364]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-09-17T11:59:20.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:19 smithi129 ceph-mon[33364]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T11:59:22.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:21 smithi049 ceph-mon[40223]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:59:22.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:21 smithi049 ceph-mon[30017]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:59:22.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:21 smithi129 ceph-mon[33364]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T11:59:24.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:23 smithi049 ceph-mon[40223]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:59:24.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:23 smithi049 ceph-mon[30017]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:59:24.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:23 smithi129 ceph-mon[33364]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-17T11:59:26.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:25 smithi049 ceph-mon[40223]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-17T11:59:26.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:25 smithi049 ceph-mon[30017]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-17T11:59:26.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:25 smithi129 ceph-mon[33364]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-17T11:59:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:28.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:27 smithi049 ceph-mon[40223]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:28.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:27 smithi049 ceph-mon[30017]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:28.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:27 smithi129 ceph-mon[33364]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:28.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:29.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:28 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:29.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:28 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:29.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:28 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:30.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:29 smithi049 ceph-mon[40223]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:30.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:29 smithi049 ceph-mon[30017]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:30.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:29 smithi129 ceph-mon[33364]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:32.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:31 smithi049 ceph-mon[40223]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:32.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:31 smithi049 ceph-mon[30017]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:32.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:31 smithi129 ceph-mon[33364]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:34.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:33 smithi049 ceph-mon[30017]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:34.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:33 smithi049 ceph-mon[40223]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:34.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:33 smithi129 ceph-mon[33364]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:36.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:35 smithi049 ceph-mon[40223]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:36.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:35 smithi049 ceph-mon[30017]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:36.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:35 smithi129 ceph-mon[33364]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:38.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:37 smithi049 ceph-mon[40223]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:38.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:37 smithi049 ceph-mon[30017]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:38.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:37 smithi129 ceph-mon[33364]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:38.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:40.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:39 smithi049 ceph-mon[40223]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:40.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:39 smithi049 ceph-mon[30017]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:39 smithi129 ceph-mon[33364]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:42.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:41 smithi049 ceph-mon[40223]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:42.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:41 smithi049 ceph-mon[30017]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:42.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:41 smithi129 ceph-mon[33364]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:43 smithi049 ceph-mon[40223]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:44.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:43 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:43 smithi049 ceph-mon[30017]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:44.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:43 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:44.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:43 smithi129 ceph-mon[33364]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T11:59:44.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:43 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:44.503 INFO:tasks.workunit.client.0.smithi049.stderr:++ seq 1 10 2024-09-17T11:59:44.504 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.504 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj1 /etc/passwd 2024-09-17T11:59:44.549 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.549 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj2 /etc/passwd 2024-09-17T11:59:44.590 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.590 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj3 /etc/passwd 2024-09-17T11:59:44.629 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.629 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj4 /etc/passwd 2024-09-17T11:59:44.666 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.666 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj5 /etc/passwd 2024-09-17T11:59:44.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.703 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj6 /etc/passwd 2024-09-17T11:59:44.740 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.740 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj7 /etc/passwd 2024-09-17T11:59:44.777 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.777 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj8 /etc/passwd 2024-09-17T11:59:44.814 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.814 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj9 /etc/passwd 2024-09-17T11:59:44.850 INFO:tasks.workunit.client.0.smithi049.stderr:+ for f in `seq 1 10` 2024-09-17T11:59:44.851 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p 2c90288b-f52d-4a05-8035-9647849d2552 put obj10 /etc/passwd 2024-09-17T11:59:44.889 INFO:tasks.workunit.client.0.smithi049.stderr:+ sleep 30 2024-09-17T11:59:46.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:45 smithi049 ceph-mon[30017]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:46.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:45 smithi049 ceph-mon[40223]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:46.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:45 smithi129 ceph-mon[33364]: pgmap v1221: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:48.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:47 smithi049 ceph-mon[40223]: pgmap v1222: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:47 smithi049 ceph-mon[30017]: pgmap v1222: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:47 smithi129 ceph-mon[33364]: pgmap v1222: 188 pgs: 188 active+clean; 759 KiB data, 813 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T11:59:48.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[30017]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-17T11:59:50.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[30017]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (reached quota's max_objects: 10) 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[30017]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[30017]: osdmap e763: 8 total, 8 up, 8 in 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[40223]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[40223]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (reached quota's max_objects: 10) 2024-09-17T11:59:50.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[40223]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:59:50.152 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:49 smithi049 ceph-mon[40223]: osdmap e763: 8 total, 8 up, 8 in 2024-09-17T11:59:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:49 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T11:59:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:49 smithi129 ceph-mon[33364]: pgmap v1223: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-17T11:59:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:49 smithi129 ceph-mon[33364]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (reached quota's max_objects: 10) 2024-09-17T11:59:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:49 smithi129 ceph-mon[33364]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-17T11:59:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:49 smithi129 ceph-mon[33364]: osdmap e763: 8 total, 8 up, 8 in 2024-09-17T11:59:52.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:51 smithi049 ceph-mon[30017]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:52.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:51 smithi049 ceph-mon[40223]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:52.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:51 smithi129 ceph-mon[33364]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:54.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:53 smithi049 ceph-mon[40223]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:54.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:53 smithi049 ceph-mon[30017]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:54.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:53 smithi129 ceph-mon[33364]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:56.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:55 smithi049 ceph-mon[40223]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:56.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:55 smithi049 ceph-mon[30017]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:56.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:55 smithi129 ceph-mon[33364]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:56.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 11:59:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:11:59:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T11:59:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[40223]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:58.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[30017]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:59:58.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:57 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:57 smithi129 ceph-mon[33364]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-17T11:59:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-17T11:59:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-17T11:59:58.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:57 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-17T11:59:58.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:57 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:58.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 11:59:58 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T11:59:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:58 smithi049 ceph-mon[30017]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:59.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:58 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:58 smithi049 ceph-mon[40223]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:59.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:58 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T11:59:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:58 smithi129 ceph-mon[33364]: from='mgr.14619 ' entity='mgr.y' 2024-09-17T11:59:59.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:58 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:59 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:00.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 11:59:59 smithi049 ceph-mon[40223]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:59 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:00.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 11:59:59 smithi049 ceph-mon[30017]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:00.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:59 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:00.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 11:59:59 smithi129 ceph-mon[33364]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:01.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[40223]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (running out of quota) 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T12:00:01.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-17T12:00:01.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:00 smithi049 ceph-mon[30017]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (running out of quota) 2024-09-17T12:00:01.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-17T12:00:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-17T12:00:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-17T12:00:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-17T12:00:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-17T12:00:01.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:00 smithi129 ceph-mon[33364]: pool '2c90288b-f52d-4a05-8035-9647849d2552' is full (running out of quota) 2024-09-17T12:00:02.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:01 smithi049 ceph-mon[40223]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-09-17T12:00:02.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:01 smithi049 ceph-mon[30017]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-09-17T12:00:02.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:01 smithi129 ceph-mon[33364]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-09-17T12:00:04.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:03 smithi129 ceph-mon[33364]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:04.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:03 smithi049 ceph-mon[40223]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:04.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:03 smithi049 ceph-mon[30017]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:06.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:05 smithi129 ceph-mon[33364]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:06.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:05 smithi049 ceph-mon[40223]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:06.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:05 smithi049 ceph-mon[30017]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:06.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:12:00:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T12:00:08.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:07 smithi129 ceph-mon[33364]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:08.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:07 smithi049 ceph-mon[40223]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:08.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:07 smithi049 ceph-mon[30017]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:08.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:00:08 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T12:00:10.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:09 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:10.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:09 smithi129 ceph-mon[33364]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:09 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:10.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:09 smithi049 ceph-mon[40223]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:10.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:09 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:10.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:09 smithi049 ceph-mon[30017]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:12.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:11 smithi129 ceph-mon[33364]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:12.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:11 smithi049 ceph-mon[40223]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:12.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:11 smithi049 ceph-mon[30017]: pgmap v1235: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:13.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:12 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:13.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:12 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:13.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:12 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:14.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:13 smithi129 ceph-mon[33364]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:14.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:13 smithi049 ceph-mon[30017]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:14.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:13 smithi049 ceph-mon[40223]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:14.890 INFO:tasks.workunit.client.0.smithi049.stderr:+ rados -p b9582b3b-e4bf-449f-bb78-cbf577bc7b0a put threemore /etc/passwd 2024-09-17T12:00:14.937 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_bytes 0 2024-09-17T12:00:15.036 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 -- 172.21.15.49:0/2746095790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 msgr2=0x7fc738104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:15.036 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 --2- 172.21.15.49:0/2746095790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc738104080 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fc730009900 tx=0x7fc73002a3f0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.036 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 -- 172.21.15.49:0/2746095790 shutdown_connections 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 --2- 172.21.15.49:0/2746095790 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc738104080 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 --2- 172.21.15.49:0/2746095790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc738101c30 0x7fc738103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 --2- 172.21.15.49:0/2746095790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc73806aa50 0x7fc7381016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 -- 172.21.15.49:0/2746095790 >> 172.21.15.49:0/2746095790 conn(0x7fc738077fe0 msgr2=0x7fc7380783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 -- 172.21.15.49:0/2746095790 shutdown_connections 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.026+0000 7fc73fadd640 1 -- 172.21.15.49:0/2746095790 wait complete. 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 Processor -- start 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 -- start start 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 0x7fc73819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc7381a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc738111320 con 0x7fc73806aa50 2024-09-17T12:00:15.037 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381111a0 con 0x7fc738101c30 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73fadd640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fc7381114a0 con 0x7fc738102070 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d879640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73e07a640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc7381a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d879640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39664/0 (socket says 172.21.15.49:39664) 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d879640 1 -- 172.21.15.49:0/1097117118 learned_addr learned my addr 172.21.15.49:0/1097117118 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d078640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 0x7fc73819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d879640 1 -- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 msgr2=0x7fc7381a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.027+0000 7fc73d879640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc7381a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73d879640 1 -- 172.21.15.49:0/1097117118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 msgr2=0x7fc73819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:15.038 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73d879640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 0x7fc73819b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73d879640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7381a0bc0 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73e07a640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc7381a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73d078640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 0x7fc73819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73d879640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c920 tx=0x7fc73400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734007c70 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7381a0eb0 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fc734007e10 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734018700 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.028+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7381a7750 con 0x7fc73806aa50 2024-09-17T12:00:15.039 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.029+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7381016f0 con 0x7fc73806aa50 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.030+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fc73400f450 con 0x7fc73806aa50 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.030+0000 7fc72affd640 1 --2- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 0x7fc72407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.030+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(763..763 src has 1..763) ==== 8146+0+0 (secure 0 0 0) 0x7fc73409e840 con 0x7fc73806aa50 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.030+0000 7fc73d078640 1 --2- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 0x7fc72407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.031+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=764}) -- 0x7fc724086a40 con 0x7fc73806aa50 2024-09-17T12:00:15.040 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.031+0000 7fc73d078640 1 --2- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 0x7fc72407d290 secure :-1 s=READY pgs=3910 cs=0 l=1 rev1=1 crypto rx=0x7fc738105290 tx=0x7fc72c009500 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:15.041 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.035+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc734014030 con 0x7fc73806aa50 2024-09-17T12:00:15.222 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.220+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} v 0) -- 0x7fc73819bc40 con 0x7fc73806aa50 2024-09-17T12:00:15.920 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.918+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v764) ==== 217+0+0 (secure 0 0 0) 0x7fc734069810 con 0x7fc73806aa50 2024-09-17T12:00:15.921 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.918+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 8 ==== osd_map(764..764 src has 1..764) ==== 628+0+0 (secure 0 0 0) 0x7fc73409c4f0 con 0x7fc73806aa50 2024-09-17T12:00:15.921 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:15.918+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=765}) -- 0x7fc7240879e0 con 0x7fc73806aa50 2024-09-17T12:00:16.043 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.041+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} v 0) -- 0x7fc7380ffb70 con 0x7fc73806aa50 2024-09-17T12:00:16.203 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:15 smithi049 ceph-mon[40223]: pgmap v1237: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:16.203 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:15 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:16.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:15 smithi049 ceph-mon[30017]: pgmap v1237: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:16.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:15 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:15 smithi129 ceph-mon[33364]: pgmap v1237: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:16.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:15 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:16.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:12:00:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T12:00:16.937 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.934+0000 7fc72affd640 1 -- 172.21.15.49:0/1097117118 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v765) ==== 217+0+0 (secure 0 0 0) 0x7fc73406e6c0 con 0x7fc73806aa50 2024-09-17T12:00:16.937 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_bytes = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 msgr2=0x7fc72407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 0x7fc72407d290 secure :-1 s=READY pgs=3910 cs=0 l=1 rev1=1 crypto rx=0x7fc738105290 tx=0x7fc72c009500 comp rx=0 tx=0).stop 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 msgr2=0x7fc73819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c920 tx=0x7fc73400cde0 comp rx=0 tx=0).stop 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 shutdown_connections 2024-09-17T12:00:16.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc72407add0 0x7fc72407d290 unknown :-1 s=CLOSED pgs=3910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc738102070 0x7fc7381a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc738101c30 0x7fc73819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 --2- 172.21.15.49:0/1097117118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc73806aa50 0x7fc73819ab20 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 >> 172.21.15.49:0/1097117118 conn(0x7fc738077fe0 msgr2=0x7fc73810e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 shutdown_connections 2024-09-17T12:00:16.941 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:16.938+0000 7fc73fadd640 1 -- 172.21.15.49:0/1097117118 wait complete. 2024-09-17T12:00:16.957 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool set-quota b9582b3b-e4bf-449f-bb78-cbf577bc7b0a max_objects 0 2024-09-17T12:00:17.049 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 -- 172.21.15.49:0/1552490986 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 msgr2=0x7fc4d010f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:17.049 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1552490986 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d010f4d0 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009be0 tx=0x7fc4c802a6c0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 -- 172.21.15.49:0/1552490986 shutdown_connections 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1552490986 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d010f4d0 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1552490986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1552490986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d0103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 -- 172.21.15.49:0/1552490986 >> 172.21.15.49:0/1552490986 conn(0x7fc4d00fd630 msgr2=0x7fc4d00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 -- 172.21.15.49:0/1552490986 shutdown_connections 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 -- 172.21.15.49:0/1552490986 wait complete. 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.045+0000 7fc4d769b640 1 Processor -- start 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 -- start start 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d01a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:17.050 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d01123f0 con 0x7fc4d0104520 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0112270 con 0x7fc4d0103c00 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d769b640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d0112570 con 0x7fc4d010d050 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d5c38640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d01a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39684/0 (socket says 172.21.15.49:39684) 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 -- 172.21.15.49:0/1120795904 learned_addr learned my addr 172.21.15.49:0/1120795904 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d5437640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 -- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 msgr2=0x7fc4d01a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d01a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 -- 172.21.15.49:0/1120795904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 msgr2=0x7fc4d019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d01a0b30 con 0x7fc4d0104520 2024-09-17T12:00:17.051 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d5437640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d5c38640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.046+0000 7fc4d4c36640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000e9c0 tx=0x7fc4c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.047+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000cd80 con 0x7fc4d0104520 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.047+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fc4c00108b0 con 0x7fc4d0104520 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.047+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d01a0e20 con 0x7fc4d0104520 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.047+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0018730 con 0x7fc4d0104520 2024-09-17T12:00:17.052 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.047+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d01a76c0 con 0x7fc4d0104520 2024-09-17T12:00:17.053 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.048+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d006b9a0 con 0x7fc4d0104520 2024-09-17T12:00:17.053 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.048+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fc4c001f020 con 0x7fc4d0104520 2024-09-17T12:00:17.053 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.049+0000 7fc4c67fc640 1 --2- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 0x7fc4bc07d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:17.054 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.049+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(765..765 src has 1..765) ==== 8146+0+0 (secure 0 0 0) 0x7fc4c009e6c0 con 0x7fc4d0104520 2024-09-17T12:00:17.054 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.049+0000 7fc4d5437640 1 --2- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 0x7fc4bc07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:17.054 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.049+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=766}) -- 0x7fc4bc086ac0 con 0x7fc4d0104520 2024-09-17T12:00:17.054 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.049+0000 7fc4d5437640 1 --2- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 0x7fc4bc07d290 secure :-1 s=READY pgs=3911 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00abe0 tx=0x7fc4cc00a5c0 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:17.055 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.053+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fc4c0069710 con 0x7fc4d0104520 2024-09-17T12:00:17.242 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.240+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} v 0) -- 0x7fc4d019bbb0 con 0x7fc4d0104520 2024-09-17T12:00:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[30017]: osdmap e764: 8 total, 8 up, 8 in 2024-09-17T12:00:17.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:17.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[40223]: osdmap e764: 8 total, 8 up, 8 in 2024-09-17T12:00:17.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:16 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:17.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:16 smithi129 ceph-mon[33364]: osdmap e764: 8 total, 8 up, 8 in 2024-09-17T12:00:17.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:16 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"} : dispatch 2024-09-17T12:00:17.938 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.936+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v766) ==== 221+0+0 (secure 0 0 0) 0x7fc4c006e5c0 con 0x7fc4d0104520 2024-09-17T12:00:17.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.938+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 8 ==== osd_map(766..766 src has 1..766) ==== 628+0+0 (secure 0 0 0) 0x7fc4c00d5190 con 0x7fc4d0104520 2024-09-17T12:00:17.940 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:17.938+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=767}) -- 0x7fc4bc087a00 con 0x7fc4d0104520 2024-09-17T12:00:18.060 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.058+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} v 0) -- 0x7fc4d01a7890 con 0x7fc4d0104520 2024-09-17T12:00:18.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:17 smithi129 ceph-mon[33364]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:18.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:18.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:17 smithi129 ceph-mon[33364]: osdmap e765: 8 total, 8 up, 8 in 2024-09-17T12:00:18.328 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:17 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:18.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[30017]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:18.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:18.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[30017]: osdmap e765: 8 total, 8 up, 8 in 2024-09-17T12:00:18.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[40223]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1097117118' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_bytes", "val": "0"}]': finished 2024-09-17T12:00:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[40223]: osdmap e765: 8 total, 8 up, 8 in 2024-09-17T12:00:18.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:17 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:18.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:00:18 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T12:00:18.955 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.952+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a v767) ==== 221+0+0 (secure 0 0 0) 0x7fc4c009c370 con 0x7fc4d0104520 2024-09-17T12:00:18.955 INFO:tasks.workunit.client.0.smithi049.stderr:set-quota max_objects = 0 for pool b9582b3b-e4bf-449f-bb78-cbf577bc7b0a 2024-09-17T12:00:18.955 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.952+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 <== mon.0 v2:172.21.15.49:3300/0 10 ==== osd_map(767..767 src has 1..767) ==== 628+0+0 (secure 0 0 0) 0x7fc4c0010430 con 0x7fc4d0104520 2024-09-17T12:00:18.955 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.952+0000 7fc4c67fc640 1 -- 172.21.15.49:0/1120795904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=768}) -- 0x7fc4bc087c50 con 0x7fc4d0104520 2024-09-17T12:00:18.959 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 msgr2=0x7fc4bc07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:18.959 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 0x7fc4bc07d290 secure :-1 s=READY pgs=3911 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00abe0 tx=0x7fc4cc00a5c0 comp rx=0 tx=0).stop 2024-09-17T12:00:18.959 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 msgr2=0x7fc4d019afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:18.959 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000e9c0 tx=0x7fc4c000ee80 comp rx=0 tx=0).stop 2024-09-17T12:00:18.959 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 shutdown_connections 2024-09-17T12:00:18.960 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> v2:172.21.15.49:6800/1163045599 conn(0x7fc4bc07add0 0x7fc4bc07d290 secure :-1 s=CLOSED pgs=3911 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00abe0 tx=0x7fc4cc00a5c0 comp rx=0 tx=0).stop 2024-09-17T12:00:18.960 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fc4d010d050 0x7fc4d01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:18.960 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d0104520 0x7fc4d019afd0 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:18.960 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 --2- 172.21.15.49:0/1120795904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d0103c00 0x7fc4d019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:18.960 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 >> 172.21.15.49:0/1120795904 conn(0x7fc4d00fd630 msgr2=0x7fc4d010f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:18.961 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.957+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 shutdown_connections 2024-09-17T12:00:18.961 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:18.958+0000 7fc4d769b640 1 -- 172.21.15.49:0/1120795904 wait complete. 2024-09-17T12:00:18.973 INFO:tasks.workunit.client.0.smithi049.stderr:+ sleep 30 2024-09-17T12:00:19.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:19.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[40223]: osdmap e766: 8 total, 8 up, 8 in 2024-09-17T12:00:19.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[30017]: osdmap e766: 8 total, 8 up, 8 in 2024-09-17T12:00:19.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:18 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:19.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:18 smithi129 ceph-mon[33364]: osdmap e766: 8 total, 8 up, 8 in 2024-09-17T12:00:19.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:18 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"} : dispatch 2024-09-17T12:00:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:20.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[30017]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[30017]: osdmap e767: 8 total, 8 up, 8 in 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[40223]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:20.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:19 smithi049 ceph-mon[40223]: osdmap e767: 8 total, 8 up, 8 in 2024-09-17T12:00:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:19 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:19 smithi129 ceph-mon[33364]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-17T12:00:20.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:19 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/1120795904' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "field": "max_objects", "val": "0"}]': finished 2024-09-17T12:00:20.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:19 smithi129 ceph-mon[33364]: osdmap e767: 8 total, 8 up, 8 in 2024-09-17T12:00:22.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:21 smithi049 ceph-mon[40223]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-17T12:00:22.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:21 smithi049 ceph-mon[30017]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-17T12:00:22.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:21 smithi129 ceph-mon[33364]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-17T12:00:24.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:23 smithi049 ceph-mon[40223]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-17T12:00:24.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:23 smithi049 ceph-mon[30017]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-17T12:00:24.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:23 smithi129 ceph-mon[33364]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-17T12:00:26.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:25 smithi049 ceph-mon[30017]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-17T12:00:26.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:25 smithi049 ceph-mon[40223]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-17T12:00:26.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:25 smithi129 ceph-mon[33364]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-17T12:00:26.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:26 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:12:00:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T12:00:28.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:27 smithi049 ceph-mon[30017]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-09-17T12:00:28.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:27 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:27 smithi049 ceph-mon[40223]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-09-17T12:00:28.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:27 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:27 smithi129 ceph-mon[33364]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2024-09-17T12:00:28.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:27 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:28.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:00:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T12:00:30.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:29 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:30.401 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:29 smithi049 ceph-mon[40223]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:29 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:30.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:29 smithi049 ceph-mon[30017]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:30.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:29 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:30.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:29 smithi129 ceph-mon[33364]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:32.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:31 smithi049 ceph-mon[30017]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-09-17T12:00:32.402 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:31 smithi049 ceph-mon[40223]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-09-17T12:00:32.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:31 smithi129 ceph-mon[33364]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-09-17T12:00:34.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:33 smithi049 ceph-mon[40223]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:34.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:33 smithi049 ceph-mon[30017]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:34.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:33 smithi129 ceph-mon[33364]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:36.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:35 smithi049 ceph-mon[40223]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:36.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:35 smithi049 ceph-mon[30017]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:36.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:35 smithi129 ceph-mon[33364]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:36.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:36 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:12:00:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T12:00:38.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:37 smithi049 ceph-mon[40223]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:38.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:37 smithi049 ceph-mon[30017]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:38.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:37 smithi129 ceph-mon[33364]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:38.926 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:00:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T12:00:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:39 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:40.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:39 smithi129 ceph-mon[33364]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:39 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:40.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:39 smithi049 ceph-mon[40223]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:39 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:40.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:39 smithi049 ceph-mon[30017]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:42.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:41 smithi049 ceph-mon[40223]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:42.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:41 smithi049 ceph-mon[30017]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:42.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:41 smithi129 ceph-mon[33364]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:43.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:42 smithi049 ceph-mon[40223]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:43.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:42 smithi049 ceph-mon[30017]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:43.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:42 smithi129 ceph-mon[33364]: from='mgr.14619 172.21.15.49:0/3931743748' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-17T12:00:44.400 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:43 smithi049 ceph-mon[40223]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:44.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:43 smithi049 ceph-mon[30017]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:44.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:43 smithi129 ceph-mon[33364]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:45.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:45 smithi049 ceph-mon[40223]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:45.901 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:45 smithi049 ceph-mon[30017]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:45.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:45 smithi129 ceph-mon[33364]: pgmap v1256: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:46.900 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:46 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: ::ffff:172.21.15.129 - - [17/Sep/2024:12:00:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-17T12:00:48.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:47 smithi049 ceph-mon[30017]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:48.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:47 smithi049 ceph-mon[40223]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:48.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:47 smithi129 ceph-mon[33364]: pgmap v1257: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T12:00:48.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:00:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug there is no tcmu-runner data available 2024-09-17T12:00:48.974 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool delete b9582b3b-e4bf-449f-bb78-cbf577bc7b0a b9582b3b-e4bf-449f-bb78-cbf577bc7b0a --yes-i-really-really-mean-it 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 -- 172.21.15.49:0/37879958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0107f70 msgr2=0x7fd6d010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 --2- 172.21.15.49:0/37879958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0107f70 0x7fd6d010e770 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009900 tx=0x7fd6c802a410 comp rx=0 tx=0).stop 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 -- 172.21.15.49:0/37879958 shutdown_connections 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 --2- 172.21.15.49:0/37879958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0107f70 0x7fd6d010e770 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 --2- 172.21.15.49:0/37879958 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 --2- 172.21.15.49:0/37879958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 -- 172.21.15.49:0/37879958 >> 172.21.15.49:0/37879958 conn(0x7fd6d00fd630 msgr2=0x7fd6d00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 -- 172.21.15.49:0/37879958 shutdown_connections 2024-09-17T12:00:49.077 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.069+0000 7fd6d7840640 1 -- 172.21.15.49:0/37879958 wait complete. 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 Processor -- start 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 -- start start 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0107f70 0x7fd6d0111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d006ba40 con 0x7fd6d0101bf0 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d006b8c0 con 0x7fd6d0107f70 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.070+0000 7fd6d7840640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d006bbc0 con 0x7fd6d01075f0 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d4ddb640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d4ddb640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3301/0 says I am v2:172.21.15.49:57838/0 (socket says 172.21.15.49:57838) 2024-09-17T12:00:49.078 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46454/0 (socket says 172.21.15.49:46454) 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d4ddb640 1 -- 172.21.15.49:0/2247245299 learned_addr learned my addr 172.21.15.49:0/2247245299 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d5ddd640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0107f70 0x7fd6d0111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 -- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 msgr2=0x7fd6d0111200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 -- 172.21.15.49:0/2247245299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0107f70 msgr2=0x7fd6d0111740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0107f70 0x7fd6d0111740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a6d00 con 0x7fd6d0101bf0 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d4ddb640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d55dc640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e9f0 tx=0x7fd6cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00cd80 con 0x7fd6d0101bf0 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7fd6cc0118f0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a6ff0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc010880 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.071+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a7500 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.072+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0101fd0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.073+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7fd6cc011470 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.074+0000 7fd6c67fc640 1 --2- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 0x7fd6bc07d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.074+0000 7fd6d4ddb640 1 --2- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 0x7fd6bc07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.074+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(767..767 src has 1..767) ==== 8146+0+0 (secure 0 0 0) 0x7fd6cc09eb00 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.074+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=768}) -- 0x7fd6bc086ac0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.074+0000 7fd6d4ddb640 1 --2- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 0x7fd6bc07d290 secure :-1 s=READY pgs=3912 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00095f0 tx=0x7fd6c0005f90 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:49.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.079+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7fd6cc016030 con 0x7fd6d0101bf0 2024-09-17T12:00:49.268 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.266+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6d01a7790 con 0x7fd6d0101bf0 2024-09-17T12:00:49.854 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.852+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true}]=0 pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' removed v768) ==== 248+0+0 (secure 0 0 0) 0x7fd6cc069b50 con 0x7fd6d0101bf0 2024-09-17T12:00:49.855 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.853+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 8 ==== osd_map(768..768 src has 1..768) ==== 296+0+0 (secure 0 0 0) 0x7fd6cc09c7b0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.855 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.853+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=769}) -- 0x7fd6bc0877e0 con 0x7fd6d0101bf0 2024-09-17T12:00:49.977 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.975+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6d0108810 con 0x7fd6d0101bf0 2024-09-17T12:00:49.978 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.976+0000 7fd6c67fc640 1 -- 172.21.15.49:0/2247245299 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true}]=0 pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' does not exist v768) ==== 255+0+0 (secure 0 0 0) 0x7fd6cc06ea00 con 0x7fd6d0101bf0 2024-09-17T12:00:49.978 INFO:tasks.workunit.client.0.smithi049.stderr:pool 'b9582b3b-e4bf-449f-bb78-cbf577bc7b0a' does not exist 2024-09-17T12:00:49.980 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 msgr2=0x7fd6bc07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 0x7fd6bc07d290 secure :-1 s=READY pgs=3912 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00095f0 tx=0x7fd6c0005f90 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 msgr2=0x7fd6d0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e9f0 tx=0x7fd6cc00eeb0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 shutdown_connections 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> v2:172.21.15.49:6800/1163045599 conn(0x7fd6bc07add0 0x7fd6bc07d290 unknown :-1 s=CLOSED pgs=3912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0107f70 0x7fd6d0111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7fd6d01075f0 0x7fd6d0111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 --2- 172.21.15.49:0/2247245299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd6d0101bf0 0x7fd6d0110cc0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 >> 172.21.15.49:0/2247245299 conn(0x7fd6d00fd630 msgr2=0x7fd6d00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 shutdown_connections 2024-09-17T12:00:49.981 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:49.978+0000 7fd6d7840640 1 -- 172.21.15.49:0/2247245299 wait complete. 2024-09-17T12:00:49.989 INFO:tasks.workunit.client.0.smithi049.stderr:+ ceph osd pool delete 2c90288b-f52d-4a05-8035-9647849d2552 2c90288b-f52d-4a05-8035-9647849d2552 --yes-i-really-really-mean-it 2024-09-17T12:00:50.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- 172.21.15.49:0/2077229056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938109070 msgr2=0x7f193810f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:50.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 --2- 172.21.15.49:0/2077229056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938109070 0x7f193810f870 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f1930009ca0 tx=0x7f193002a8c0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.079 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- 172.21.15.49:0/2077229056 shutdown_connections 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 --2- 172.21.15.49:0/2077229056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938109070 0x7f193810f870 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 --2- 172.21.15.49:0/2077229056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 --2- 172.21.15.49:0/2077229056 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938102cf0 0x7f19381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- 172.21.15.49:0/2077229056 >> 172.21.15.49:0/2077229056 conn(0x7f19380fe750 msgr2=0x7f1938100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- 172.21.15.49:0/2077229056 shutdown_connections 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- 172.21.15.49:0/2077229056 wait complete. 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 Processor -- start 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.075+0000 7f193f08c640 1 -- start start 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 0x7f193819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:50.080 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1938112440 con 0x7f1938102cf0 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19381122c0 con 0x7f19381086f0 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 -- --> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] -- mon_getmap magic: 0 -- 0x7f19381125c0 con 0x7f1938109070 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193d629640 1 --2- >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 0x7f193819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46480/0 (socket says 172.21.15.49:46480) 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 -- 172.21.15.49:0/3431964033 learned_addr learned my addr 172.21.15.49:0/3431964033 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f1937fff640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 -- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 msgr2=0x7f193819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:50.081 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 0x7f193819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 -- 172.21.15.49:0/3431964033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 msgr2=0x7f1938198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819ea50 con 0x7f1938102cf0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193d629640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 0x7f193819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f1937fff640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193ce28640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e9f0 tx=0x7f192c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00cd80 con 0x7f1938102cf0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(25 keys) ==== 1054+0+0 (secure 0 0 0) 0x7f192c0054a0 con 0x7f1938102cf0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819ed40 con 0x7f1938102cf0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.076+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c010430 con 0x7f1938102cf0 2024-09-17T12:00:50.082 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.077+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a5400 con 0x7f1938102cf0 2024-09-17T12:00:50.083 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.078+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19381030d0 con 0x7f1938102cf0 2024-09-17T12:00:50.083 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.078+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 103101+0+0 (secure 0 0 0) 0x7f192c005020 con 0x7f1938102cf0 2024-09-17T12:00:50.083 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.079+0000 7f1935ffb640 1 --2- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 0x7f192807d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T12:00:50.084 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.079+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(768..768 src has 1..768) ==== 7758+0+0 (secure 0 0 0) 0x7f192c09e940 con 0x7f1938102cf0 2024-09-17T12:00:50.084 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.079+0000 7f1937fff640 1 --2- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 0x7f192807d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T12:00:50.084 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.079+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=769}) -- 0x7f19280862a0 con 0x7f1938102cf0 2024-09-17T12:00:50.084 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.079+0000 7f1937fff640 1 --2- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 0x7f192807d290 secure :-1 s=READY pgs=3913 cs=0 l=1 rev1=1 crypto rx=0x7f1924006fd0 tx=0x7f1924008040 comp rx=0 tx=0).ready entity=mgr.14619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T12:00:50.085 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.083+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196987 (secure 0 0 0) 0x7f192c069be0 con 0x7f1938102cf0 2024-09-17T12:00:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[40223]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[40223]: pgmap v1258: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:50.150 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[30017]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[30017]: pgmap v1258: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:50.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:49 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:50.176 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:49 smithi129 ceph-mon[33364]: from='client.24463 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T12:00:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:49 smithi129 ceph-mon[33364]: pgmap v1258: 188 pgs: 188 active+clean; 897 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T12:00:50.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:49 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:50.272 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.270+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19381a56c0 con 0x7f1938102cf0 2024-09-17T12:00:50.867 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.864+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true}]=0 pool '2c90288b-f52d-4a05-8035-9647849d2552' removed v769) ==== 248+0+0 (secure 0 0 0) 0x7f192c06ea90 con 0x7f1938102cf0 2024-09-17T12:00:50.867 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.864+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 8 ==== osd_map(769..769 src has 1..769) ==== 296+0+0 (secure 0 0 0) 0x7f192c09c780 con 0x7f1938102cf0 2024-09-17T12:00:50.867 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.864+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=770}) -- 0x7f1928086d20 con 0x7f1938102cf0 2024-09-17T12:00:50.985 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.982+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19381a5b10 con 0x7f1938102cf0 2024-09-17T12:00:50.985 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.983+0000 7f1935ffb640 1 -- 172.21.15.49:0/3431964033 <== mon.0 v2:172.21.15.49:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true}]=0 pool '2c90288b-f52d-4a05-8035-9647849d2552' does not exist v769) ==== 255+0+0 (secure 0 0 0) 0x7f192c06c0c0 con 0x7f1938102cf0 2024-09-17T12:00:50.985 INFO:tasks.workunit.client.0.smithi049.stderr:pool '2c90288b-f52d-4a05-8035-9647849d2552' does not exist 2024-09-17T12:00:50.987 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 msgr2=0x7f192807d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 0x7f192807d290 secure :-1 s=READY pgs=3913 cs=0 l=1 rev1=1 crypto rx=0x7f1924006fd0 tx=0x7f1924008040 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 msgr2=0x7f19381989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e9f0 tx=0x7f192c00edf0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 shutdown_connections 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> v2:172.21.15.49:6800/1163045599 conn(0x7f192807add0 0x7f192807d290 unknown :-1 s=CLOSED pgs=3913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] conn(0x7f1938109070 0x7f193819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19381086f0 0x7f1938198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 --2- 172.21.15.49:0/3431964033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1938102cf0 0x7f19381989b0 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 >> 172.21.15.49:0/3431964033 conn(0x7f19380fe750 msgr2=0x7f1938069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 shutdown_connections 2024-09-17T12:00:50.988 INFO:tasks.workunit.client.0.smithi049.stderr:2024-09-17T12:00:50.985+0000 7f193f08c640 1 -- 172.21.15.49:0/3431964033 wait complete. 2024-09-17T12:00:50.996 INFO:tasks.workunit.client.0.smithi049.stdout:OK 2024-09-17T12:00:50.997 INFO:tasks.workunit.client.0.smithi049.stderr:+ echo OK 2024-09-17T12:00:50.998 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-17T12:00:50.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-17T12:00:51.066 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-17T12:00:51.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-17T12:00:51.129 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:51.129 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[40223]: osdmap e768: 8 total, 8 up, 8 in 2024-09-17T12:00:51.129 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.129 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[30017]: osdmap e768: 8 total, 8 up, 8 in 2024-09-17T12:00:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.130 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:50 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:51.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:50 smithi129 ceph-mon[33364]: osdmap e768: 8 total, 8 up, 8 in 2024-09-17T12:00:51.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/2247245299' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "pool2": "b9582b3b-e4bf-449f-bb78-cbf577bc7b0a", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.177 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:50 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:51.421 DEBUG:teuthology.parallel:result is None 2024-09-17T12:00:51.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T12:00:51.487 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T12:00:51.487 DEBUG:teuthology.orchestra.run.smithi049:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-17T12:00:51.581 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T12:00:51.581 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T12:00:51.590 INFO:tasks.cephadm:Teardown begin 2024-09-17T12:00:51.590 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T12:00:51.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T12:00:51.650 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T12:00:51.651 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T12:00:51.673 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T12:00:51.708 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T12:00:51.708 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T12:00:51.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a 2024-09-17T12:00:51.926 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:51 smithi129 ceph-mon[33364]: pgmap v1260: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:51 smithi129 ceph-mon[33364]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T12:00:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:51 smithi129 ceph-mon[33364]: osdmap e769: 8 total, 8 up, 8 in 2024-09-17T12:00:51.927 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:51 smithi129 ceph-mon[33364]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:52.003 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[40223]: pgmap v1260: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:52.003 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[40223]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T12:00:52.003 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:52.003 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[40223]: osdmap e769: 8 total, 8 up, 8 in 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[40223]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 systemd[1]: Stopping Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[30017]: pgmap v1260: 176 pgs: 176 active+clean; 708 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[30017]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true}]': finished 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[30017]: osdmap e769: 8 total, 8 up, 8 in 2024-09-17T12:00:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:51 smithi049 ceph-mon[30017]: from='client.? 172.21.15.49:0/3431964033' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "2c90288b-f52d-4a05-8035-9647849d2552", "pool2": "2c90288b-f52d-4a05-8035-9647849d2552", "yes_i_really_really_mean_it": true} : dispatch 2024-09-17T12:00:52.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a[29994]: 2024-09-17T12:00:52.000+0000 7f1476f74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:00:52.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a[29994]: 2024-09-17T12:00:52.000+0000 7f1476f74640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T12:00:52.401 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 podman[146773]: 2024-09-17 12:00:52.072867512 +0000 UTC m=+0.174454570 container died b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-7498493, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-17T12:00:52.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 podman[146773]: 2024-09-17 12:00:52.690772187 +0000 UTC m=+0.792359245 container cleanup b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=main-7498493, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-17T12:00:52.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 bash[146773]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a 2024-09-17T12:00:53.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T12:00:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:52 smithi049 podman[146790]: 2024-09-17 12:00:52.89279655 +0000 UTC m=+0.816215353 container remove b088646d0261e5b48b41751d9d3b52ece41f2a2a70f32fa8b5c8853a125c4c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-17T12:00:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:53 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T12:00:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:53 smithi049 systemd[1]: Stopped Ceph mon.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:00:53.150 INFO:journalctl@ceph.mon.a.smithi049.stdout:Sep 17 12:00:53 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.a.service: Consumed 38.838s CPU time. 2024-09-17T12:00:53.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:00:53.168 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T12:00:53.169 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-17T12:00:53.169 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.c 2024-09-17T12:00:53.515 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:53 smithi049 systemd[1]: Stopping Ceph mon.c for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c[40200]: 2024-09-17T12:00:53.513+0000 7ff4df43d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:00:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c[40200]: 2024-09-17T12:00:53.513+0000 7ff4df43d640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T12:00:53.900 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:53 smithi049 podman[146905]: 2024-09-17 12:00:53.739419533 +0000 UTC m=+0.338174567 container died 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE Bus STOPPING 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE Bus STOPPED 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE Bus STARTING 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE Serving on http://:::9283 2024-09-17T12:00:53.901 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:53 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: [17/Sep/2024:12:00:53] ENGINE Bus STARTED 2024-09-17T12:00:54.248 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:54 smithi049 podman[146905]: 2024-09-17 12:00:54.012802963 +0000 UTC m=+0.611557998 container cleanup 01fb6ff14ea79180e5e410b227a836147dd7ba3093fcf83c67e4333f7d31ae34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-7498493, CEPH_POINT_RELEASE=, ceph=True) 2024-09-17T12:00:54.248 INFO:journalctl@ceph.mon.c.smithi049.stdout:Sep 17 12:00:54 smithi049 bash[146905]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-c 2024-09-17T12:00:54.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T12:00:54.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:00:54.433 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-17T12:00:54.433 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-17T12:00:54.433 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.b 2024-09-17T12:00:54.838 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:54 smithi129 systemd[1]: Stopping Ceph mon.b for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:55.128 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b[33342]: 2024-09-17T12:00:54.835+0000 7f34b729c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:00:55.128 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b[33342]: 2024-09-17T12:00:54.835+0000 7f34b729c640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T12:00:55.128 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:54 smithi129 podman[67409]: 2024-09-17 12:00:54.856717075 +0000 UTC m=+0.221806522 container died 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-17T12:00:55.426 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:55 smithi129 podman[67409]: 2024-09-17 12:00:55.309458517 +0000 UTC m=+0.674547963 container cleanup 5ef188ccb6c587666ddcaf57525bb44019c67c4de23c90092d10eaeae26b55e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-17T12:00:55.427 INFO:journalctl@ceph.mon.b.smithi129.stdout:Sep 17 12:00:55 smithi129 bash[67409]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mon-b 2024-09-17T12:00:55.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T12:00:55.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:00:55.842 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-17T12:00:55.843 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T12:00:55.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y 2024-09-17T12:00:56.150 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:55 smithi049 systemd[1]: Stopping Ceph mgr.y for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:56.528 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T12:00:56.243+0000 7fba48a37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:00:56.528 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:56 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y[30268]: 2024-09-17T12:00:56.243+0000 7fba48a37640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-17T12:00:56.528 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:56 smithi049 podman[147049]: 2024-09-17 12:00:56.275760648 +0000 UTC m=+0.235149041 container died d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=main-7498493) 2024-09-17T12:00:56.820 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:56 smithi049 podman[147049]: 2024-09-17 12:00:56.592509273 +0000 UTC m=+0.551897670 container cleanup d7621ffdb924d6c0a6ee54d381169db0ffc242126282d6160a0ec7882b73dba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7498493279982dfe87f00616198a5967475f6169, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-7498493, org.label-schema.license=GPLv2) 2024-09-17T12:00:56.820 INFO:journalctl@ceph.mgr.y.smithi049.stdout:Sep 17 12:00:56 smithi049 bash[147049]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-y 2024-09-17T12:00:56.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T12:00:56.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:00:56.988 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T12:00:56.989 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T12:00:56.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.x 2024-09-17T12:00:57.315 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 12:00:57 smithi129 systemd[1]: Stopping Ceph mgr.x for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:57.600 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 12:00:57 smithi129 podman[67541]: 2024-09-17 12:00:57.314316669 +0000 UTC m=+0.143573536 container died 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-17T12:00:57.926 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 12:00:57 smithi129 podman[67541]: 2024-09-17 12:00:57.68119189 +0000 UTC m=+0.510448747 container cleanup 954ae3af2a7ffec268f544915c909be470688049c203e9f5b0268aeed14ee106 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=main-7498493, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:00:57.926 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 17 12:00:57 smithi129 bash[67541]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-mgr-x 2024-09-17T12:00:58.082 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T12:00:58.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:00:58.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T12:00:58.116 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T12:00:58.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.0 2024-09-17T12:00:58.400 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:00:58 smithi049 systemd[1]: Stopping Ceph osd.0 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:00:58.900 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:00:58 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T12:00:58.441+0000 7f94bc2e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:00:58.900 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:00:58 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T12:00:58.441+0000 7f94bc2e4640 -1 osd.0 769 *** Got signal Terminated *** 2024-09-17T12:00:58.901 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:00:58 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0[43854]: 2024-09-17T12:00:58.441+0000 7f94bc2e4640 -1 osd.0 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:04.276 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:03 smithi049 podman[147185]: 2024-09-17 12:01:03.945702302 +0000 UTC m=+5.632611168 container died 6df1d0af136d422cbb4820ff86a502e4ba9d382c856bcdbc45ec075888b05fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:04.591 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:04 smithi049 podman[147185]: 2024-09-17 12:01:04.363834162 +0000 UTC m=+6.050743011 container cleanup 6df1d0af136d422cbb4820ff86a502e4ba9d382c856bcdbc45ec075888b05fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-17T12:01:04.592 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:04 smithi049 bash[147185]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0 2024-09-17T12:01:04.900 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:04 smithi049 podman[147219]: 2024-09-17 12:01:04.590344254 +0000 UTC m=+0.636781011 container remove 6df1d0af136d422cbb4820ff86a502e4ba9d382c856bcdbc45ec075888b05fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-17T12:01:05.303 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:05.033244392 +0000 UTC m=+0.104170245 container create d560d06cceb78f24bfd501a5fd124ffb040a226b96eacb332ac226e174d27475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-17T12:01:05.304 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:04.945812509 +0000 UTC m=+0.016738341 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:05.304 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:05.302614952 +0000 UTC m=+0.373540778 container init d560d06cceb78f24bfd501a5fd124ffb040a226b96eacb332ac226e174d27475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-17T12:01:05.650 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:05.307860419 +0000 UTC m=+0.378786239 container start d560d06cceb78f24bfd501a5fd124ffb040a226b96eacb332ac226e174d27475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:05.651 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:05.402570292 +0000 UTC m=+0.473496118 container attach d560d06cceb78f24bfd501a5fd124ffb040a226b96eacb332ac226e174d27475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T12:01:05.651 INFO:journalctl@ceph.osd.0.smithi049.stdout:Sep 17 12:01:05 smithi049 podman[147313]: 2024-09-17 12:01:05.504353044 +0000 UTC m=+0.575278868 container died d560d06cceb78f24bfd501a5fd124ffb040a226b96eacb332ac226e174d27475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-7498493, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T12:01:06.081 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T12:01:06.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:06.124 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T12:01:06.124 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T12:01:06.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.1 2024-09-17T12:01:06.400 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:06 smithi049 systemd[1]: Stopping Ceph osd.1 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:06.900 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T12:01:06.461+0000 7f08d365a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:06.901 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T12:01:06.461+0000 7f08d365a640 -1 osd.1 769 *** Got signal Terminated *** 2024-09-17T12:01:06.901 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:06 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1[50043]: 2024-09-17T12:01:06.461+0000 7f08d365a640 -1 osd.1 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:12.262 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:11 smithi049 podman[147440]: 2024-09-17 12:01:11.948813402 +0000 UTC m=+5.597638734 container died 4af4a8e79363f9a7b757e325d20f5193dd90f3f959bc3d1215b40b764782c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-7498493, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-17T12:01:12.578 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:12 smithi049 podman[147440]: 2024-09-17 12:01:12.367336593 +0000 UTC m=+6.016161914 container cleanup 4af4a8e79363f9a7b757e325d20f5193dd90f3f959bc3d1215b40b764782c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=main-7498493, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:12.578 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:12 smithi049 bash[147440]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1 2024-09-17T12:01:12.900 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:12 smithi049 podman[147465]: 2024-09-17 12:01:12.577046089 +0000 UTC m=+0.624816998 container remove 4af4a8e79363f9a7b757e325d20f5193dd90f3f959bc3d1215b40b764782c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T12:01:13.306 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:12 smithi049 podman[147559]: 2024-09-17 12:01:12.899633408 +0000 UTC m=+0.016423677 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:13.306 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147559]: 2024-09-17 12:01:13.002738726 +0000 UTC m=+0.119529019 container create cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-7498493, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:13.306 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147559]: 2024-09-17 12:01:13.221562909 +0000 UTC m=+0.338353170 container init cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-7498493, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:13.306 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147559]: 2024-09-17 12:01:13.226752715 +0000 UTC m=+0.343542961 container start cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=main-7498493) 2024-09-17T12:01:13.306 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147559]: 2024-09-17 12:01:13.304701457 +0000 UTC m=+0.421491724 container attach cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:13.650 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147559]: 2024-09-17 12:01:13.417507773 +0000 UTC m=+0.534298039 container died cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-7498493, io.buildah.version=1.37.2) 2024-09-17T12:01:13.983 INFO:journalctl@ceph.osd.1.smithi049.stdout:Sep 17 12:01:13 smithi049 podman[147602]: 2024-09-17 12:01:13.966565101 +0000 UTC m=+0.539206561 container remove cfb28db01c1cef6a17a037d27d84ba787ad94eb131fa2401ded0f3738b1a2bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=main-7498493, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-17T12:01:13.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T12:01:14.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:14.031 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T12:01:14.031 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T12:01:14.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.2 2024-09-17T12:01:14.356 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:14 smithi049 systemd[1]: Stopping Ceph osd.2 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:14.650 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:14 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T12:01:14.353+0000 7f34bd0c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:14.650 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:14 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T12:01:14.353+0000 7f34bd0c6640 -1 osd.2 769 *** Got signal Terminated *** 2024-09-17T12:01:14.650 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:14 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2[55760]: 2024-09-17T12:01:14.353+0000 7f34bd0c6640 -1 osd.2 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:20.150 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:19 smithi049 podman[147687]: 2024-09-17 12:01:19.898545048 +0000 UTC m=+5.637209462 container died a17853cf441f801cf150c6cc89be90bac44ef83a5865c469cbe766f52719705e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-17T12:01:20.531 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:20 smithi049 podman[147687]: 2024-09-17 12:01:20.302778587 +0000 UTC m=+6.041443006 container cleanup a17853cf441f801cf150c6cc89be90bac44ef83a5865c469cbe766f52719705e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-7498493, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:20.531 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:20 smithi049 bash[147687]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2 2024-09-17T12:01:20.879 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:20 smithi049 podman[147708]: 2024-09-17 12:01:20.529401376 +0000 UTC m=+0.622216708 container remove a17853cf441f801cf150c6cc89be90bac44ef83a5865c469cbe766f52719705e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T12:01:21.150 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:20 smithi049 podman[147812]: 2024-09-17 12:01:20.783126662 +0000 UTC m=+0.018485326 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:21.150 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:20 smithi049 podman[147812]: 2024-09-17 12:01:20.880143265 +0000 UTC m=+0.115501913 container create e1939146b79b7e299bacc48304a3daf51b2345d879bcf43a68f5e5180cd08d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-17T12:01:21.150 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:21 smithi049 podman[147812]: 2024-09-17 12:01:21.083048941 +0000 UTC m=+0.318407618 container init e1939146b79b7e299bacc48304a3daf51b2345d879bcf43a68f5e5180cd08d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-17T12:01:21.151 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:21 smithi049 podman[147812]: 2024-09-17 12:01:21.088895018 +0000 UTC m=+0.324253667 container start e1939146b79b7e299bacc48304a3daf51b2345d879bcf43a68f5e5180cd08d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=main-7498493, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T12:01:21.459 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:21 smithi049 podman[147812]: 2024-09-17 12:01:21.199502605 +0000 UTC m=+0.434861266 container attach e1939146b79b7e299bacc48304a3daf51b2345d879bcf43a68f5e5180cd08d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:21.459 INFO:journalctl@ceph.osd.2.smithi049.stdout:Sep 17 12:01:21 smithi049 podman[147812]: 2024-09-17 12:01:21.288936204 +0000 UTC m=+0.524294868 container died e1939146b79b7e299bacc48304a3daf51b2345d879bcf43a68f5e5180cd08d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-17T12:01:21.912 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T12:01:21.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:21.952 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T12:01:21.952 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-17T12:01:21.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.3 2024-09-17T12:01:22.150 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:22 smithi049 systemd[1]: Stopping Ceph osd.3 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:22.652 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:22 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T12:01:22.264+0000 7ff210301640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:22.653 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:22 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T12:01:22.264+0000 7ff210301640 -1 osd.3 769 *** Got signal Terminated *** 2024-09-17T12:01:22.653 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:22 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T12:01:22.264+0000 7ff210301640 -1 osd.3 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:25.848 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:25.505+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:26.176 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:25 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:25.982+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:26.864 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:26 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:26.466+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:27.117 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:26 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:26.947+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:27.400 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:27 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3[61541]: 2024-09-17T12:01:27.107+0000 7ff20c916640 -1 osd.3 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:00.031480+0000 front 2024-09-17T12:01:00.031522+0000 (oldest deadline 2024-09-17T12:01:26.432107+0000) 2024-09-17T12:01:27.898 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:27 smithi049 podman[147940]: 2024-09-17 12:01:27.593528298 +0000 UTC m=+5.417415408 container died fd3435064d12377757dd25c6efa28e0722fb21544209d459c0dd42f75ed4d256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=main-7498493, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T12:01:27.898 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:27 smithi049 podman[147940]: 2024-09-17 12:01:27.896485615 +0000 UTC m=+5.720372712 container cleanup fd3435064d12377757dd25c6efa28e0722fb21544209d459c0dd42f75ed4d256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-7498493, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:27.916 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:27.477+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:27.917 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:27.499+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:28.150 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:27 smithi049 bash[147940]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3 2024-09-17T12:01:28.150 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[147961]: 2024-09-17 12:01:28.089611055 +0000 UTC m=+0.493403930 container remove fd3435064d12377757dd25c6efa28e0722fb21544209d459c0dd42f75ed4d256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=main-7498493) 2024-09-17T12:01:28.177 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:27 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:27.915+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:28.177 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:28.138+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:28.693 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.406867773 +0000 UTC m=+0.104941035 container create 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T12:01:28.693 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.318231159 +0000 UTC m=+0.016304443 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:28.694 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.601079885 +0000 UTC m=+0.299153173 container init 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:28.694 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.605990644 +0000 UTC m=+0.304063907 container start 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-17T12:01:28.694 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.692226271 +0000 UTC m=+0.390299542 container attach 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-17T12:01:28.880 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:28.549+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:28.881 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:28.501+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:29.113 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:28 smithi049 podman[148054]: 2024-09-17 12:01:28.800390107 +0000 UTC m=+0.498463382 container died 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=main-7498493, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-17T12:01:29.142 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:28 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:28.878+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:29.377 INFO:journalctl@ceph.osd.3.smithi049.stdout:Sep 17 12:01:29 smithi049 podman[148097]: 2024-09-17 12:01:29.353568046 +0000 UTC m=+0.541936754 container remove 3e8b6963c883b9c7b830d116f05b12ad9d5ba1c9d6ec201de741bf49088cffed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=main-7498493, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T12:01:29.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.3.service' 2024-09-17T12:01:29.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:29.425 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-17T12:01:29.425 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-17T12:01:29.425 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.4 2024-09-17T12:01:29.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:29.140+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:29.740 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:29 smithi129 systemd[1]: Stopping Ceph osd.4 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:29.740 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:29.544+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:29.741 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:29.573+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.618Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.621Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.621Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.621Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.622Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.624Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:29.741 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:01:29.624Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.49:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.49:8765: connect: connection refused" 2024-09-17T12:01:30.148 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:29.738+0000 7ff477ae4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:30.149 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:29.738+0000 7ff477ae4640 -1 osd.4 769 *** Got signal Terminated *** 2024-09-17T12:01:30.149 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:29.738+0000 7ff477ae4640 -1 osd.4 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:30.149 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:29 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:29.862+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:30.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:30 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:30.146+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:30.816 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:30 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:30.598+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:30.817 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:30 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:30.494+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:31.141 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:30 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:30.814+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:31.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:31 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:31.140+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:31.861 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:31 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:31.584+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:31.861 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:31 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:31.520+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:32.172 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:31 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:31.859+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:32.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:32.170+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:32.837 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:32.603+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:32.837 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:32.551+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:33.176 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:32.835+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:33.177 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:32.835+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.134233+0000 front 2024-09-17T12:01:08.133991+0000 (oldest deadline 2024-09-17T12:01:32.233894+0000) 2024-09-17T12:01:33.541 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:33.196+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:33.541 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:33.197+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:33.805 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:33.641+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:33.806 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:33.641+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:33.806 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:33.539+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:33.806 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:33.539+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:34.176 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:33.803+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:03.132433+0000 front 2024-09-17T12:01:03.132519+0000 (oldest deadline 2024-09-17T12:01:25.432385+0000) 2024-09-17T12:01:34.177 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:33 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4[37887]: 2024-09-17T12:01:33.803+0000 7ff4700f1640 -1 osd.4 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.134233+0000 front 2024-09-17T12:01:08.133991+0000 (oldest deadline 2024-09-17T12:01:32.233894+0000) 2024-09-17T12:01:34.503 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:34.216+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:34.504 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:34.216+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:34.926 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:34.615+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:34.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:34.615+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:34.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:34.502+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:34.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:34 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:34.502+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:35.540 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:35 smithi129 podman[67702]: 2024-09-17 12:01:35.240887057 +0000 UTC m=+5.613763850 container died 3317fce654b3b5fcf5e22846f6d816e8e1c7642112165b206492f3a439689d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=main-7498493, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-17T12:01:35.540 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:35.245+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:35.540 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:35.245+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:35.540 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:35.481+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:35.541 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:35.481+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:35.865 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:35.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:35.865 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:35 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:35.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:35.865 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:35 smithi129 podman[67702]: 2024-09-17 12:01:35.637110567 +0000 UTC m=+6.009987361 container cleanup 3317fce654b3b5fcf5e22846f6d816e8e1c7642112165b206492f3a439689d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:35.865 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:35 smithi129 bash[67702]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4 2024-09-17T12:01:36.176 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:35 smithi129 podman[67721]: 2024-09-17 12:01:35.863767756 +0000 UTC m=+0.621777877 container remove 3317fce654b3b5fcf5e22846f6d816e8e1c7642112165b206492f3a439689d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-17T12:01:36.483 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:36.231+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:36.483 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:36.231+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:36.483 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.087107123 +0000 UTC m=+0.015987143 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:36.484 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.197926459 +0000 UTC m=+0.126806460 container create 09b84789517c1021f9c86b4dd5164b9fb1aed6b5f7995f15884c19eb23133dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-7498493, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-17T12:01:36.484 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.408837903 +0000 UTC m=+0.337717912 container init 09b84789517c1021f9c86b4dd5164b9fb1aed6b5f7995f15884c19eb23133dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T12:01:36.484 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.413501575 +0000 UTC m=+0.342381577 container start 09b84789517c1021f9c86b4dd5164b9fb1aed6b5f7995f15884c19eb23133dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-7498493) 2024-09-17T12:01:36.897 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:36.591+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:36.897 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:36.591+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:36.898 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.495817932 +0000 UTC m=+0.424697942 container attach 09b84789517c1021f9c86b4dd5164b9fb1aed6b5f7995f15884c19eb23133dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-7498493, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-17T12:01:36.898 INFO:journalctl@ceph.osd.4.smithi129.stdout:Sep 17 12:01:36 smithi129 podman[67815]: 2024-09-17 12:01:36.606845069 +0000 UTC m=+0.535725079 container died 09b84789517c1021f9c86b4dd5164b9fb1aed6b5f7995f15884c19eb23133dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-17T12:01:36.898 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:36.481+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:36.898 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:36 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:36.481+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:37.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.4.service' 2024-09-17T12:01:37.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:37.255 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-17T12:01:37.256 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-17T12:01:37.256 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.5 2024-09-17T12:01:37.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:37 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:37.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused" 2024-09-17T12:01:37.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:37 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:37.018Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:01:37.502 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 systemd[1]: Stopping Ceph osd.5 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:37.503 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:37.260+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:37.503 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:37.260+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:37.626+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:37.626+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:37.501+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:37.501+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:37.592+0000 7f9dc138f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:37.592+0000 7f9dc138f640 -1 osd.5 769 *** Got signal Terminated *** 2024-09-17T12:01:37.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:37 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:37.592+0000 7f9dc138f640 -1 osd.5 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:38.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:38.262+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:38.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:38.262+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:38.926 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:38.515+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:38.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:38.515+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:38.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:38.603+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:38.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:38 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:38.603+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:39.540 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:39.241+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:39.540 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:39.241+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:39.926 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:39.538+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:39.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:39.538+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:39.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:39.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:39.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:39 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:39.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:40.528 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:40.218+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:40.528 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:40.218+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:40.529 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:40.526+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:40.529 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:40.526+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:40.926 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:40.636+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:40.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:40 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:40.636+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:41.543 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:41.200+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:41.543 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:41.200+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:41.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:41.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:41.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:41.628+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:41.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:41.541+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:41.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:41 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:41.541+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:42.571 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:42.244+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:42.571 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:42.244+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:42.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:42.670+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:42.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:42.670+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:42.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:42.569+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:00:59.204607+0000 front 2024-09-17T12:00:59.204643+0000 (oldest deadline 2024-09-17T12:01:24.504424+0000) 2024-09-17T12:01:42.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:42.569+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:08.505877+0000 front 2024-09-17T12:01:08.505917+0000 (oldest deadline 2024-09-17T12:01:32.605726+0000) 2024-09-17T12:01:42.927 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:42 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5[43827]: 2024-09-17T12:01:42.569+0000 7f9db999c640 -1 osd.5 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:17.906713+0000 front 2024-09-17T12:01:17.906695+0000 (oldest deadline 2024-09-17T12:01:42.006807+0000) 2024-09-17T12:01:43.327 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:43 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:43.237+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:43.327 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:43 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:43.237+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:43.327 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:43 smithi129 podman[67944]: 2024-09-17 12:01:43.045836816 +0000 UTC m=+5.552030603 container died 88255b477a274c3e936a269c628fd8fc45abb8a995cfb2158a602fd0006081d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-17T12:01:43.676 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:43 smithi129 podman[67944]: 2024-09-17 12:01:43.424991902 +0000 UTC m=+5.931185689 container cleanup 88255b477a274c3e936a269c628fd8fc45abb8a995cfb2158a602fd0006081d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=main-7498493) 2024-09-17T12:01:43.677 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:43 smithi129 bash[67944]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5 2024-09-17T12:01:44.038 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:43 smithi129 podman[67966]: 2024-09-17 12:01:43.700962625 +0000 UTC m=+0.649529254 container remove 88255b477a274c3e936a269c628fd8fc45abb8a995cfb2158a602fd0006081d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-17T12:01:44.039 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:43 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:43.707+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:44.039 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:43 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:43.707+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:44.347 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:43.94041245 +0000 UTC m=+0.015788181 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:44.058998497 +0000 UTC m=+0.134374228 container create 9ea6ce6605519ef95a04ff6368c172dec2cf2c62afbc5a2e0dbbd50cf07caba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-7498493, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:44.262768252 +0000 UTC m=+0.338143990 container init 9ea6ce6605519ef95a04ff6368c172dec2cf2c62afbc5a2e0dbbd50cf07caba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=main-7498493, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:44.267394575 +0000 UTC m=+0.342770309 container start 9ea6ce6605519ef95a04ff6368c172dec2cf2c62afbc5a2e0dbbd50cf07caba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:44.346244841 +0000 UTC m=+0.421620576 container attach 9ea6ce6605519ef95a04ff6368c172dec2cf2c62afbc5a2e0dbbd50cf07caba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-7498493, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:44.259+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:44.259+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:44.348 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:44.259+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:44.676 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 17 12:01:44 smithi129 podman[68059]: 2024-09-17 12:01:44.460269495 +0000 UTC m=+0.535645227 container died 9ea6ce6605519ef95a04ff6368c172dec2cf2c62afbc5a2e0dbbd50cf07caba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-7498493, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-17T12:01:45.034 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:44.677+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:45.034 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:44 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:44.677+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:45.054 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.5.service' 2024-09-17T12:01:45.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:45.093 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-17T12:01:45.093 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-17T12:01:45.093 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.6 2024-09-17T12:01:45.422 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 systemd[1]: Stopping Ceph osd.6 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.280+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.280+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.280+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.420+0000 7f3d0e5cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.420+0000 7f3d0e5cb640 -1 osd.6 769 *** Got signal Terminated *** 2024-09-17T12:01:45.423 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:45.420+0000 7f3d0e5cb640 -1 osd.6 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:45.676 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:45.633+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:45.677 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:45.633+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:45.677 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:45 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:45.633+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:46.676 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:46.277+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:46.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:46.277+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:46.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:46.277+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:47.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:46.680+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:47.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:46.680+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:47.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:46.680+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:47.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:46 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:46.681+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:47.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:47 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:47.018Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:01:47.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:47 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:47.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused" 2024-09-17T12:01:47.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:47 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:47.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:01:47.636 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:47.240+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:47.636 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:47.240+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:47.636 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:47.240+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:47.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:47.633+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:47.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:47.634+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:47.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:47.634+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:47.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:47 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:47.634+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:48.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:48.249+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:48.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:48.249+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:48.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:48.249+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:49.176 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:48.676+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:49.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:48.676+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:49.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:48.676+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:49.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:48 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:48.676+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:49.643 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:49.243+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:49.643 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:49.243+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:49.643 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:49.243+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:49.643 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:49.243+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:25.702268+0000 front 2024-09-17T12:01:25.702202+0000 (oldest deadline 2024-09-17T12:01:49.202078+0000) 2024-09-17T12:01:49.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:49.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:49.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:49.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:49.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:49.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:49.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:49.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:49.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:49 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:49.642+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:50.676 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:50.269+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:02.999344+0000 front 2024-09-17T12:01:02.999206+0000 (oldest deadline 2024-09-17T12:01:27.699119+0000) 2024-09-17T12:01:50.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:50.269+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:07.699692+0000 front 2024-09-17T12:01:07.699659+0000 (oldest deadline 2024-09-17T12:01:32.999581+0000) 2024-09-17T12:01:50.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:50.269+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:19.301381+0000 front 2024-09-17T12:01:19.301340+0000 (oldest deadline 2024-09-17T12:01:43.401277+0000) 2024-09-17T12:01:50.677 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6[49464]: 2024-09-17T12:01:50.269+0000 7f3d0abe0640 -1 osd.6 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:25.702268+0000 front 2024-09-17T12:01:25.702202+0000 (oldest deadline 2024-09-17T12:01:49.202078+0000) 2024-09-17T12:01:51.163 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:50 smithi129 podman[68189]: 2024-09-17 12:01:50.892603821 +0000 UTC m=+5.577951517 container died fd3c0bf8aa04a524b7d73e781afa9d11833fc1cb48bfeb80fd71aa7f14dcc294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T12:01:51.164 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:50.683+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:51.164 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:50.683+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:51.164 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:50.683+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:51.164 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:50.683+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:51.164 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:50 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:50.683+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:51.426 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:51 smithi129 podman[68189]: 2024-09-17 12:01:51.261075724 +0000 UTC m=+5.946423405 container cleanup fd3c0bf8aa04a524b7d73e781afa9d11833fc1cb48bfeb80fd71aa7f14dcc294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:51.427 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:51 smithi129 bash[68189]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:51 smithi129 podman[68210]: 2024-09-17 12:01:51.554608341 +0000 UTC m=+0.656343901 container remove fd3c0bf8aa04a524b7d73e781afa9d11833fc1cb48bfeb80fd71aa7f14dcc294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6, RELEASE=main-7498493, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:51 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:51.640+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:51 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:51.640+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:51 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:51.640+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:51 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:51.640+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:51.883 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:51 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:51.641+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:52.159 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:51 smithi129 podman[68303]: 2024-09-17 12:01:51.784363092 +0000 UTC m=+0.016165955 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:01:52.159 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:51 smithi129 podman[68303]: 2024-09-17 12:01:51.913889224 +0000 UTC m=+0.145692153 container create 12102ee56c48276cda9bb20bcb63551036985bc1a761f41208962235209add7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=main-7498493, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-17T12:01:52.159 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:52 smithi129 podman[68303]: 2024-09-17 12:01:52.158106459 +0000 UTC m=+0.389909332 container init 12102ee56c48276cda9bb20bcb63551036985bc1a761f41208962235209add7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-7498493, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T12:01:52.427 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:52 smithi129 podman[68303]: 2024-09-17 12:01:52.163026397 +0000 UTC m=+0.394829263 container start 12102ee56c48276cda9bb20bcb63551036985bc1a761f41208962235209add7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-7498493, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:52.427 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:52 smithi129 podman[68303]: 2024-09-17 12:01:52.249697937 +0000 UTC m=+0.481500815 container attach 12102ee56c48276cda9bb20bcb63551036985bc1a761f41208962235209add7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-17T12:01:52.427 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 17 12:01:52 smithi129 podman[68303]: 2024-09-17 12:01:52.351312971 +0000 UTC m=+0.583115834 container died 12102ee56c48276cda9bb20bcb63551036985bc1a761f41208962235209add7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:01:53.046 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:52 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:52.685+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:53.046 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:52 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:52.685+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:53.046 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:52 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:52.685+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:53.046 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:52 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:52.685+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:53.047 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:52 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:52.685+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:53.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.6.service' 2024-09-17T12:01:53.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:01:53.106 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-17T12:01:53.106 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-17T12:01:53.106 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.7 2024-09-17T12:01:53.426 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 systemd[1]: Stopping Ceph osd.7 for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:01:53.724 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.432+0000 7f53bb36c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:01:53.724 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.432+0000 7f53bb36c640 -1 osd.7 769 *** Got signal Terminated *** 2024-09-17T12:01:53.724 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.432+0000 7f53bb36c640 -1 osd.7 769 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T12:01:54.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.721+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:54.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.721+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:54.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.721+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:54.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.721+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:54.177 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:53 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:53.722+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:55.078 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:54.688+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:55.078 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:54.688+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:55.078 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:54.688+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:55.078 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:54.688+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:55.079 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:54 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:54.688+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.129:6802 osd.4 since back 2024-09-17T12:01:29.476293+0000 front 2024-09-17T12:01:29.476213+0000 (oldest deadline 2024-09-17T12:01:55.376241+0000) 2024-09-17T12:01:55.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:55 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:55.642+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:56.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:56.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:56.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:56.928 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:56.928 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.129:6802 osd.4 since back 2024-09-17T12:01:29.476293+0000 front 2024-09-17T12:01:29.476213+0000 (oldest deadline 2024-09-17T12:01:55.376241+0000) 2024-09-17T12:01:56.928 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:56 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:56.652+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:57.311 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:57 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:57.019Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:01:57.311 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:57 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:57.019Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused" 2024-09-17T12:01:57.311 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:01:57 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:01:57.020Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6803 osd.0 since back 2024-09-17T12:01:01.472789+0000 front 2024-09-17T12:01:01.472907+0000 (oldest deadline 2024-09-17T12:01:27.372642+0000) 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6807 osd.1 since back 2024-09-17T12:01:11.373949+0000 front 2024-09-17T12:01:11.374034+0000 (oldest deadline 2024-09-17T12:01:33.574461+0000) 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6811 osd.2 since back 2024-09-17T12:01:13.574960+0000 front 2024-09-17T12:01:13.575129+0000 (oldest deadline 2024-09-17T12:01:44.775379+0000) 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.49:6815 osd.3 since back 2024-09-17T12:01:24.775826+0000 front 2024-09-17T12:01:24.775848+0000 (oldest deadline 2024-09-17T12:01:49.475898+0000) 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 heartbeat_check: no reply from 172.21.15.129:6802 osd.4 since back 2024-09-17T12:01:29.476293+0000 front 2024-09-17T12:01:29.476213+0000 (oldest deadline 2024-09-17T12:01:55.376241+0000) 2024-09-17T12:01:57.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:57 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7[55138]: 2024-09-17T12:01:57.665+0000 7f53b3979640 -1 osd.7 769 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24460.0:1147 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e769) 2024-09-17T12:01:59.134 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:58 smithi129 podman[68432]: 2024-09-17 12:01:58.84482492 +0000 UTC m=+5.515915183 container died 3d879d6bc58708c906d12ed43635a807609808fd634f5fe6a1efeb6cdde3de3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=main-7498493, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T12:01:59.426 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:59 smithi129 podman[68432]: 2024-09-17 12:01:59.231206744 +0000 UTC m=+5.902297004 container cleanup 3d879d6bc58708c906d12ed43635a807609808fd634f5fe6a1efeb6cdde3de3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7, RELEASE=main-7498493, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:01:59.426 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:59 smithi129 bash[68432]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7 2024-09-17T12:01:59.782 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:59 smithi129 podman[68465]: 2024-09-17 12:01:59.474099646 +0000 UTC m=+0.622187082 container remove 3d879d6bc58708c906d12ed43635a807609808fd634f5fe6a1efeb6cdde3de3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=main-7498493, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-17T12:02:00.103 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:59 smithi129 podman[68560]: 2024-09-17 12:01:59.684442693 +0000 UTC m=+0.015933964 image pull 6c1fcab85602ba9d6ddde3c89adee6ee0382b7527a8ed3ee372c969068775c76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb 2024-09-17T12:02:00.104 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:01:59 smithi129 podman[68560]: 2024-09-17 12:01:59.783022842 +0000 UTC m=+0.114514117 container create adc28fa912c9f89a2c2988c0bdb9fa73e59c99fae9f6343b0a3586402694640f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-17T12:02:00.104 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:02:00 smithi129 podman[68560]: 2024-09-17 12:02:00.010691035 +0000 UTC m=+0.342182300 container init adc28fa912c9f89a2c2988c0bdb9fa73e59c99fae9f6343b0a3586402694640f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T12:02:00.104 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:02:00 smithi129 podman[68560]: 2024-09-17 12:02:00.015546705 +0000 UTC m=+0.347037971 container start adc28fa912c9f89a2c2988c0bdb9fa73e59c99fae9f6343b0a3586402694640f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T12:02:00.104 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:02:00 smithi129 podman[68560]: 2024-09-17 12:02:00.102364837 +0000 UTC m=+0.433856101 container attach adc28fa912c9f89a2c2988c0bdb9fa73e59c99fae9f6343b0a3586402694640f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-7498493, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-17T12:02:00.426 INFO:journalctl@ceph.osd.7.smithi129.stdout:Sep 17 12:02:00 smithi129 podman[68560]: 2024-09-17 12:02:00.211049809 +0000 UTC m=+0.542541076 container died adc28fa912c9f89a2c2988c0bdb9fa73e59c99fae9f6343b0a3586402694640f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-7498493, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T12:02:00.809 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@osd.7.service' 2024-09-17T12:02:00.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:02:00.848 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-17T12:02:00.848 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T12:02:00.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T12:02:01.150 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:00 smithi049 systemd[1]: Stopping Ceph rgw.foo.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:01.650 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:01 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a[66224]: 2024-09-17T12:02:01.153+0000 7f3944340640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T12:02:01.650 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:01 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a[66224]: 2024-09-17T12:02:01.153+0000 7f3947bac880 -1 shutting down 2024-09-17T12:02:07.400 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:07 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:02:07.020Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused" 2024-09-17T12:02:07.401 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:07 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:02:07.020Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.49:8443: connect: connection refused" 2024-09-17T12:02:07.401 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:07 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:02:07.021Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.129:8443: connect: connection refused" 2024-09-17T12:02:11.462 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:11 smithi049 bash[148194]: time="2024-09-17T12:02:11Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T12:02:11.463 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:11 smithi049 podman[148194]: 2024-09-17 12:02:11.189925799 +0000 UTC m=+10.131677059 container died c6a5a4b09e8600d34251426abe23359149da6c1110c120e4b494d83923f144f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=main-7498493, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T12:02:11.820 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:11 smithi049 podman[148194]: 2024-09-17 12:02:11.551096706 +0000 UTC m=+10.492847965 container cleanup c6a5a4b09e8600d34251426abe23359149da6c1110c120e4b494d83923f144f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-7498493, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T12:02:11.820 INFO:journalctl@ceph.rgw.foo.a.smithi049.stdout:Sep 17 12:02:11 smithi049 bash[148194]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T12:02:11.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T12:02:12.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:02:12.034 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T12:02:12.034 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T12:02:12.035 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T12:02:12.328 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 systemd[1]: Stopping Ceph prometheus.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.326Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.327Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.328Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.328Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-17T12:02:12.602 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a[64117]: ts=2024-09-17T12:02:12.328Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-17T12:02:12.603 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 podman[68688]: 2024-09-17 12:02:12.33719374 +0000 UTC m=+0.114468950 container died e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:12.927 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 podman[68688]: 2024-09-17 12:02:12.716569905 +0000 UTC m=+0.493845110 container cleanup e8818dcb2aa25d0798f9f4f2328fd1c5dd1c83b83d2fa36c6b286f4338ece2d3 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:12.927 INFO:journalctl@ceph.prometheus.a.smithi129.stdout:Sep 17 12:02:12 smithi129 bash[68688]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T12:02:13.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T12:02:13.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T12:02:13.177 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T12:02:13.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm rm-cluster --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T12:02:13.388 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T12:02:15.317 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 systemd[1]: Stopping Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:15.317 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a[75050]: ts=2024-09-17T12:02:15.274Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T12:02:15.640 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 podman[148709]: 2024-09-17 12:02:15.316052198 +0000 UTC m=+0.119900921 container died 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:15.640 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 podman[148709]: 2024-09-17 12:02:15.639345965 +0000 UTC m=+0.443194685 container cleanup 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:15.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 bash[148709]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T12:02:15.900 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 podman[148726]: 2024-09-17 12:02:15.849047571 +0000 UTC m=+0.524288549 container remove 263ef1b944909d420073f149aac2232bbc263eb487fe768004127a2f74c46f61 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:16.365 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:15 smithi049 podman[148726]: 2024-09-17 12:02:15.924754457 +0000 UTC m=+0.599995393 volume remove 7456266d2b9bb247de954a468e1487e5d9debfbf2769c8480a0819c883928bde 2024-09-17T12:02:16.366 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:16 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T12:02:16.366 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:16 smithi049 systemd[1]: Stopped Ceph alertmanager.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:02:16.366 INFO:journalctl@ceph.alertmanager.a.smithi049.stdout:Sep 17 12:02:16 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.891s CPU time. 2024-09-17T12:02:16.616 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:16 smithi049 systemd[1]: Stopping Ceph node-exporter.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:16.900 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:16 smithi049 podman[148846]: 2024-09-17 12:02:16.615377713 +0000 UTC m=+0.107159703 container died 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:17.278 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 podman[148846]: 2024-09-17 12:02:17.076811106 +0000 UTC m=+0.568593093 container cleanup 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:17.279 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 bash[148846]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T12:02:17.582 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 podman[148864]: 2024-09-17 12:02:17.277646694 +0000 UTC m=+0.663148626 container remove 10f6ccdd6fa4264ae43827bdf1765f2e064517096ab8b4056f4f0029603db788 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T12:02:17.582 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T12:02:17.582 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T12:02:17.582 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 systemd[1]: Stopped Ceph node-exporter.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:02:17.582 INFO:journalctl@ceph.node-exporter.a.smithi049.stdout:Sep 17 12:02:17 smithi049 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 3.909s CPU time. 2024-09-17T12:02:18.065 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm rm-cluster --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T12:02:18.268 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T12:02:19.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:19 smithi129 systemd[1]: Stopping Ceph iscsi.iscsi.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:20.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:19 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a[60214]: debug Shutdown received 2024-09-17T12:02:30.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:29 smithi129 bash[69110]: time="2024-09-17T12:02:29Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-17T12:02:30.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:29 smithi129 podman[69110]: 2024-09-17 12:02:29.823694496 +0000 UTC m=+10.134186296 container died 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-7498493, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-17T12:02:30.356 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 podman[69110]: 2024-09-17 12:02:30.129023221 +0000 UTC m=+10.439515010 container cleanup 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=main-7498493, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-17T12:02:30.356 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 bash[69110]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-17T12:02:30.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 podman[69136]: 2024-09-17 12:02:30.355053412 +0000 UTC m=+0.531894087 container remove 4c5ebb4bb8c1fba846f3acca824a3864578c7726b1b1bb90fa236b0839f3b409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d1ee255599fffe6c832aeedeed0b06c298863896564017e014e180f38b9ae1bb, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=main-7498493, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T12:02:30.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T12:02:30.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-17T12:02:30.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 systemd[1]: Stopped Ceph iscsi.iscsi.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:02:30.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi129.stdout:Sep 17 12:02:30 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.724s CPU time. 2024-09-17T12:02:32.023 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:31 smithi129 systemd[1]: Stopping Ceph grafana.a for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:32.426 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=server t=2024-09-17T12:02:32.212186688Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-17T12:02:32.426 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=grafana-apiserver t=2024-09-17T12:02:32.212633507Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-17T12:02:32.427 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=tracing t=2024-09-17T12:02:32.212664183Z level=info msg="Closing tracing" 2024-09-17T12:02:32.427 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a[62835]: logger=ticker t=2024-09-17T12:02:32.212756133Z level=info msg=stopped last_tick=2024-09-17T12:02:30Z 2024-09-17T12:02:32.427 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 podman[69422]: 2024-09-17 12:02:32.222791904 +0000 UTC m=+0.138083536 container died 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T12:02:32.753 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 podman[69422]: 2024-09-17 12:02:32.551509534 +0000 UTC m=+0.466801155 container cleanup 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T12:02:32.754 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 bash[69422]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a 2024-09-17T12:02:33.166 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 podman[69442]: 2024-09-17 12:02:32.752875104 +0000 UTC m=+0.531123866 container remove 1d2e68980535cb7afa0844e24ca119c66e90420389da6cd303df3ec555e28ed8 (image=quay.io/ceph/grafana:10.4.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-17T12:02:33.166 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@grafana.a.service: Deactivated successfully. 2024-09-17T12:02:33.166 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 systemd[1]: Stopped Ceph grafana.a for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:02:33.166 INFO:journalctl@ceph.grafana.a.smithi129.stdout:Sep 17 12:02:32 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@grafana.a.service: Consumed 12.839s CPU time. 2024-09-17T12:02:33.428 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:33 smithi129 systemd[1]: Stopping Ceph node-exporter.b for f97d9e96-74e8-11ef-bceb-c7b262605968... 2024-09-17T12:02:33.680 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:33 smithi129 podman[69562]: 2024-09-17 12:02:33.427554863 +0000 UTC m=+0.123430542 container died e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T12:02:34.055 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:33 smithi129 podman[69562]: 2024-09-17 12:02:33.769922689 +0000 UTC m=+0.465798384 container cleanup e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T12:02:34.055 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:33 smithi129 bash[69562]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T12:02:34.337 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:34 smithi129 podman[69579]: 2024-09-17 12:02:34.054526488 +0000 UTC m=+0.630621864 container remove e7a705ebe7df3de66daefb66867bc0eb9bc4ea01ba3038c03f80e6213e537949 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-f97d9e96-74e8-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T12:02:34.338 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:34 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T12:02:34.338 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:34 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T12:02:34.338 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:34 smithi129 systemd[1]: Stopped Ceph node-exporter.b for f97d9e96-74e8-11ef-bceb-c7b262605968. 2024-09-17T12:02:34.338 INFO:journalctl@ceph.node-exporter.b.smithi129.stdout:Sep 17 12:02:34 smithi129 systemd[1]: ceph-f97d9e96-74e8-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 3.554s CPU time. 2024-09-17T12:02:34.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T12:02:34.853 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T12:02:34.880 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T12:02:34.882 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi049/crash 2024-09-17T12:02:34.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash -- . 2024-09-17T12:02:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T12:02:34.939 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-09-17T12:02:34.940 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi129/crash 2024-09-17T12:02:34.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash -- . 2024-09-17T12:02:34.976 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T12:02:34.976 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-09-17T12:02:34.977 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T12:02:34.977 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v CEPHADM_STRAY_DAEMON | head -n 1 2024-09-17T12:02:35.010 INFO:tasks.cephadm:Compressing logs... 2024-09-17T12:02:35.011 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T12:02:35.054 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T12:02:35.078 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/tcmu-runner.log 2024-09-17T12:02:35.078 INFO:teuthology.orchestra.run.smithi129.stderr:: No such file or directory 2024-09-17T12:02:35.079 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-17T12:02:35.079 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-17T12:02:35.079 INFO:teuthology.orchestra.run.smithi129.stderr: 63.1% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/tcmu-runner.log.gz 2024-09-17T12:02:35.079 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-17T12:02:35.079 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:: No such file or directory 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-17T12:02:35.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose 89.2% -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.y.log -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.6.log: 79.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-17T12:02:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-17T12:02:35.086 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.x.log: 87.1% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-17T12:02:35.090 INFO:teuthology.orchestra.run.smithi129.stderr: 92.5% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-17T12:02:35.090 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-17T12:02:35.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log 2024-09-17T12:02:35.092 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-17T12:02:35.092 INFO:teuthology.orchestra.run.smithi129.stderr: 91.5% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-17T12:02:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-17T12:02:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-17T12:02:35.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log: 95.3% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-17T12:02:35.103 INFO:teuthology.orchestra.run.smithi049.stderr: 92.8% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-17T12:02:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-17T12:02:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-17T12:02:35.106 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-17T12:02:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-17T12:02:35.146 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-17T12:02:35.524 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-17T12:02:36.036 INFO:teuthology.orchestra.run.smithi049.stderr: 92.4% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-17T12:02:36.445 INFO:teuthology.orchestra.run.smithi129.stderr: 92.8% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-17T12:02:37.424 INFO:teuthology.orchestra.run.smithi049.stderr: 92.1% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-17T12:02:41.206 INFO:teuthology.orchestra.run.smithi129.stderr: 93.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-17T12:02:41.427 INFO:teuthology.orchestra.run.smithi129.stderr: 93.8% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-17T12:02:41.508 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-17T12:02:41.929 INFO:teuthology.orchestra.run.smithi129.stderr: 93.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-17T12:02:42.296 INFO:teuthology.orchestra.run.smithi049.stderr: 93.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-17T12:02:42.551 INFO:teuthology.orchestra.run.smithi129.stderr: 94.0% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-17T12:02:42.553 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T12:02:42.553 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m7.485s 2024-09-17T12:02:42.553 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m27.245s 2024-09-17T12:02:42.553 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.931s 2024-09-17T12:02:42.566 INFO:teuthology.orchestra.run.smithi049.stderr: 93.9% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-17T12:02:43.252 INFO:teuthology.orchestra.run.smithi049.stderr: 94.0% -- replaced with /var/log/ceph/f97d9e96-74e8-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-17T12:02:43.254 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-09-17T12:02:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m8.187s 2024-09-17T12:02:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m32.023s 2024-09-17T12:02:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m1.034s 2024-09-17T12:02:43.255 INFO:tasks.cephadm:Archiving logs... 2024-09-17T12:02:43.255 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi049/log 2024-09-17T12:02:43.256 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T12:02:47.269 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi129/log 2024-09-17T12:02:47.270 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T12:02:51.043 INFO:tasks.cephadm:Removing cluster... 2024-09-17T12:02:51.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo cephadm rm-cluster --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 --force 2024-09-17T12:02:51.255 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T12:02:51.559 DEBUG:teuthology.orchestra.run.smithi129:> sudo cephadm rm-cluster --fsid f97d9e96-74e8-11ef-bceb-c7b262605968 --force 2024-09-17T12:02:51.754 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: f97d9e96-74e8-11ef-bceb-c7b262605968 2024-09-17T12:02:52.045 INFO:tasks.cephadm:Teardown complete 2024-09-17T12:02:52.046 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T12:02:52.055 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T12:02:52.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T12:02:52.058 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T12:02:52.136 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T12:02:52.144 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T12:02:52.145 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T12:02:52.147 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T12:02:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-09-17T12:02:52.161 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 -547us[ -542us] +/- 48ms 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 25 +1718us[+1723us] +/- 72ms 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -1064us[-1064us] +/- 74ms 2024-09-17T12:02:52.165 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -457us[ -454us] +/- 48ms 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +1782us[+1782us] +/- 72ms 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -984us[ -981us] +/- 74ms 2024-09-17T12:02:52.167 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T12:02:52.167 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T12:02:52.176 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T12:02:52.176 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T12:02:52.185 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T12:02:52.287 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T12:02:52.357 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-17T12:02:52.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T12:02:52.388 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-17T12:02:52.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T12:02:52.416 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T12:02:52.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T12:02:52.433 INFO:teuthology.task.internal:Duration was 2001.460427 seconds 2024-09-17T12:02:52.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T12:02:52.442 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T12:02:52.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T12:02:52.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T12:02:52.484 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T12:02:52.495 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T12:02:52.718 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T12:02:52.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-09-17T12:02:52.718 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T12:02:52.749 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T12:02:52.749 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T12:02:52.788 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T12:02:52.789 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T12:02:52.793 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T12:02:52.855 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T12:02:52.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T12:02:52.860 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T12:02:53.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T12:02:53.300 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T12:02:53.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T12:02:53.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T12:02:53.351 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T12:02:53.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T12:02:53.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T12:02:53.390 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-09-17T12:02:53.414 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-17T12:02:53.440 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T12:02:53.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T12:02:53.460 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T12:02:53.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T12:02:53.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T12:02:53.505 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T12:02:53.506 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi049 2024-09-17T12:02:53.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T12:02:53.568 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_20:08:15-orch-main-distro-default-smithi/7907768/remote/smithi129 2024-09-17T12:02:53.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T12:02:53.605 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T12:02:53.605 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T12:02:53.608 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T12:02:53.654 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T12:02:53.667 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T12:02:53.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T12:02:53.676 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T12:02:53.676 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T12:02:53.679 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T12:02:53.692 INFO:teuthology.orchestra.run.smithi049.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 12:02 /home/ubuntu/cephtest 2024-09-17T12:02:53.709 INFO:teuthology.orchestra.run.smithi129.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 12:02 /home/ubuntu/cephtest 2024-09-17T12:02:53.710 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T12:02:53.720 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T12:02:53.740 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} duration: 2001.4604272842407 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T12:02:53.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T12:02:53.827 INFO:teuthology.run:pass